{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "数据集只有一个文件(diabetes.csv):Pima Indians Diabetes Dataset 包括根据医疗\n",
    "记录的比马印第安人 5 年内糖尿病的发病情况,这是一个两类分类问题。每个类的\n",
    "样本数目数量不均等。一共有 768 个样本,每个样本有 8 个输入变量和 1 个输出\n",
    "变量。缺失值通常用零值编码。\n",
    "Pregnancies: 怀孕次数\n",
    "Glucose: 口服葡萄糖耐受试验中,2 小时的血浆葡萄糖浓度。\n",
    "BloodPressure: 舒张压(mm Hg)\n",
    "SkinThickness: 三头肌皮肤褶层厚度(mm)\n",
    "Insulin:2 小时血清胰岛素含量(μU/ ml)YI\n",
    "BMI: 体重指数(体重,kg /(身高,m)^ 2)\n",
    "2) DiabetesPedigreeFunction: 糖尿病家族史\n",
    "3) Age: 年龄(岁)\n",
    "二、作业要求：\n",
    "    1. 对数据做数据探索分析（可参考0_EDA_ diabetes.ipynb，不计分）\n",
    "    2. 适当的特征工程（可参考1_FE_ diabetes.ipynb，不计分）\n",
    "    3. 采用5折交叉验证，分别用log似然损失和正确率，对Logistic回归模型的正则超参数调优。（各50分）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "from sklearn.model_selection import GridSearchCV \n",
    "\n",
    "from sklearn.metrics import log_loss  \n",
    "#from sklearn.metrics import accuracy_score\n",
    "\n",
    "from matplotlib import pyplot\n",
    "import seaborn as sns\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  2.读数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Pregnancies</th>\n",
       "      <th>Glucose</th>\n",
       "      <th>BloodPressure</th>\n",
       "      <th>SkinThickness</th>\n",
       "      <th>Insulin</th>\n",
       "      <th>BMI</th>\n",
       "      <th>DiabetesPedigreeFunction</th>\n",
       "      <th>Age</th>\n",
       "      <th>Outcome</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>6</td>\n",
       "      <td>148.0</td>\n",
       "      <td>72.0</td>\n",
       "      <td>35.0</td>\n",
       "      <td>125.0</td>\n",
       "      <td>33.6</td>\n",
       "      <td>0.627</td>\n",
       "      <td>50</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>85.0</td>\n",
       "      <td>66.0</td>\n",
       "      <td>29.0</td>\n",
       "      <td>125.0</td>\n",
       "      <td>26.6</td>\n",
       "      <td>0.351</td>\n",
       "      <td>31</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>8</td>\n",
       "      <td>183.0</td>\n",
       "      <td>64.0</td>\n",
       "      <td>29.0</td>\n",
       "      <td>125.0</td>\n",
       "      <td>23.3</td>\n",
       "      <td>0.672</td>\n",
       "      <td>32</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>89.0</td>\n",
       "      <td>66.0</td>\n",
       "      <td>23.0</td>\n",
       "      <td>94.0</td>\n",
       "      <td>28.1</td>\n",
       "      <td>0.167</td>\n",
       "      <td>21</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>137.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>35.0</td>\n",
       "      <td>168.0</td>\n",
       "      <td>43.1</td>\n",
       "      <td>2.288</td>\n",
       "      <td>33</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Pregnancies  Glucose  BloodPressure  SkinThickness  Insulin   BMI  \\\n",
       "0            6    148.0           72.0           35.0    125.0  33.6   \n",
       "1            1     85.0           66.0           29.0    125.0  26.6   \n",
       "2            8    183.0           64.0           29.0    125.0  23.3   \n",
       "3            1     89.0           66.0           23.0     94.0  28.1   \n",
       "4            0    137.0           40.0           35.0    168.0  43.1   \n",
       "\n",
       "   DiabetesPedigreeFunction  Age  Outcome  \n",
       "0                     0.627   50        1  \n",
       "1                     0.351   31        0  \n",
       "2                     0.672   32        1  \n",
       "3                     0.167   21        0  \n",
       "4                     2.288   33        1  "
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 读取数据\n",
    "train = pd.read_csv(\"/home/listen/secondhomework/TheSecondWeekHomework/FEdiabetes.csv\")\n",
    "train.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#   3.分离XY"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "Y_train=train['Outcome']\n",
    "X_train=train.drop(['Outcome'],axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  4 .数据预处理"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "ss_X = StandardScaler()\n",
    "\n",
    "X_train = ss_X.fit_transform(X_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  模型训练    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "log-loss of each fold is [0.48797856 0.53011593 0.4562292  0.422546   0.48392885]\n",
      "CV loss is  0.47615970944434044\n"
     ]
    }
   ],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "lr=LogisticRegression()\n",
    "# 交叉验证用于评估模型性能和进行参数调优（模型选择）\n",
    "from sklearn.model_selection import cross_val_score\n",
    "loss=cross_val_score(lr,X_train,Y_train,cv=5,scoring='neg_log_loss')   ##分类问题用-log——loss指标更合适\n",
    "print('log-loss of each fold is',-loss)\n",
    "print('CV loss is ',-loss.mean())##取平均数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 正则化的 Logistic Regression及参数调优\n",
    "\n",
    "这部分用交叉验证GridSearchCV、LogisticRegressionCV任意一种方式均可\n",
    "\n",
    "logistic回归的需要调整超参数有：C（正则系数，一般在log域（取log后的值）均匀设置候选参数）和正则函数penalty（L2/L1） 目标函数为：J = C* sum(logloss(f(xi), yi)) + penalty\n",
    "在sklearn框架下，不同学习器的参数调整步骤相同：\n",
    "1.设置候选参数集合 2.生成一个GridSearchCV的实例 3.调用GridSearchCV的fit函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=5, error_score='raise',\n",
       "       estimator=LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
       "          intercept_scaling=1, max_iter=100, multi_class='ovr', n_jobs=1,\n",
       "          penalty='l2', random_state=None, solver='liblinear', tol=0.0001,\n",
       "          verbose=0, warm_start=False),\n",
       "       fit_params=None, iid=True, n_jobs=1,\n",
       "       param_grid={'penalty': ['l1', 'l2'], 'C': (0.001, 0.01, 0.1, 1, 10, 100, 1000)},\n",
       "       pre_dispatch='2*n_jobs', refit=True, return_train_score='warn',\n",
       "       scoring='neg_log_loss', verbose=0)"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.model_selection import GridSearchCV \n",
    "penaltys=['l1','l2']\n",
    "Cs=(0.001,0.01,0.1,1,10,100,1000)\n",
    "#tuned_parameters = dict(penalty = penaltys, C = Cs)\n",
    "tuned_parameters=dict(penalty=penaltys,C=Cs)#############################\n",
    "lr_penalty=LogisticRegression()\n",
    "grid=GridSearchCV (lr_penalty,tuned_parameters,cv=5, scoring='neg_log_loss')\n",
    "grid.fit(X_train,Y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.476027264610514\n",
      "{'C': 1, 'penalty': 'l1'}\n"
     ]
    }
   ],
   "source": [
    "print(-grid.best_score_)\n",
    "print(grid.best_params_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/listen/anaconda3/lib/python3.7/site-packages/sklearn/utils/deprecation.py:122: FutureWarning: You are accessing a training score ('mean_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "/home/listen/anaconda3/lib/python3.7/site-packages/sklearn/utils/deprecation.py:122: FutureWarning: You are accessing a training score ('std_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xd8VFX+//HXZ2bSEwiQAKEXQamhREFR7IgKoQgIipQVXSvg7mLdnwVdBSt+V1xFFJEiYKGIKEVBFKUEpHeRElowEAikZ87vj5lACIFJm9xM8nn6mEfm3jl35n0B55NbzjlijEEppZS6FJvVAZRSSpV9WiyUUkp5pMVCKaWUR1oslFJKeaTFQimllEdaLJRSSnmkxUIppZRHWiyUUkp5pMVCKaWURw6rA5SUiIgI06BBA6tjKKWUT1m7du1fxphIT+28WixEpCvwLmAHJhpjxuR5/R3gRvdiMFDdGBPufm0w8G/3a68YYyZf6rMaNGhAXFxcScZXSqlyT0T2FaSd14qFiNiB8cCtQDywRkTmGWO25rQxxjyRq/3jQFv386rAC0AMYIC17m1PeCuvUkqpi/PmNYurgN3GmD3GmAxgBtDjEu0HAJ+7n98GLDbGHHcXiMVAVy9mVUopdQneLBa1gQO5luPd6y4gIvWBhsCPhd1WKaWU93nzmoXks+5i46H3B740xmQXZlsReRB4EKBevXpFyaiUKgcyMzOJj48nLS3N6ihlVmBgIHXq1MHPz69I23uzWMQDdXMt1wEOXaRtf+DRPNvekGfbZXk3MsZMACYAxMTE6MQcSlVQ8fHxhIWF0aBBA0Ty+12zYjPGkJiYSHx8PA0bNizSe3jzNNQaoImINBQRf1wFYV7eRiJyOVAF+C3X6oVAFxGpIiJVgC7udUopdYG0tDSqVaumheIiRIRq1aoV68jLa0cWxpgsEXkM15e8HfjEGLNFREYDccaYnMIxAJhhck3ZZ4w5LiIv4yo4AKONMce9lVUp5fu0UFxacf98vNrPwhizAFiQZ93zeZZfvMi2nwCfeC2cW3ZWFmsmDqde1xHUanC5tz9OKVVG3P2h62TGzL9fbXES31Dhh/s49OcWmh+Zg9+nt7Fr40qr4yilfFRoaOjZ5127diU8PJxu3brl2/bRRx+lTZs2NG/enKCgINq0aUObNm348ssvC/WZ69at4/vvvy9W7oKq8MWibpNojvebi0Go8VUvfv/5G6sjKaV83KhRo5gyZcpFXx8/fjzr169nwYIFNG7cmPXr17N+/Xr69OlTqM/RYlHKGjS/Ehm2mCR7NZovGcqKeR9bHUkp5cNuvvlmwsLCirTtrl27uO2222jfvj2dO3dm586dAMyYMYOWLVsSHR3NjTfeSGpqKqNHj2batGlFOioprHIzkGBxRda5jNPDl7L//ViuXvtPliQd5eb7ntGLZkr5mJe+2cLWQ6c8ttt62NUm59rFpTSvVYkXurcodraCePDBB5k4cSKNGzdmxYoVPPbYYyxatIiXXnqJZcuWUaNGDZKSkggKCuL5559n8+bNjBs3zuu5tFjkEhoeSYN/LGbb+L7csmcsi8cf5oaHxuHnsFsdTSlVASQlJbFy5Uruuuuus+uysrIA6NSpE4MGDaJv37707t271LNpscjDLzCU5iPnsenD+7k14TOWvnOMmEcnERYcZHU0pVQBFPQIoCzeDWWMISIigvXr11/w2kcffcSqVauYP38+0dHRbNy4sVSz6TWLfIjdj1YPT2bLZQ9y45nv2PxOD44majcPpZR3ValShaioKGbPng2A0+lkw4YNAOzZs4eOHTvy8ssvU6VKFQ4ePEhYWBjJycmlkk2LxcWI0GLgG+yMeZEOGas5+l5Xdu/bb3UqpZQPuO666+jbty8//PADderUYeHCgg9AMWPGDD744AOio6Np0aIF8+fPB+CJJ56gVatWtGrViltuuYWWLVty0003sWHDBtq2bev1C9ySq+O0T4uJiTHemvxo/y+fU3PJYxygBkm9Z9K+dSuvfI5Sqmi2bdtGs2bNCrVNWTwN5W35/TmJyFpjTIynbfWaRQHUu3YACZUiqfn1fYR8FcvSk5O48bobrI6llCqGilQkSoKehiqg6q1vwTnkO/zt0G7JAObO/YLyclSmlFKeaLEohLAGbQh+ZCmp/tXouu5hPv/sf2Q7tWAopco/LRaFFBjRgOojlvFXaFPu3vMs099/kdSMbM8bKqWUD9NiUQS20Ahqj1jMochO3PfXOOaOe4zEZJ2hSylVfmmxKCr/EOo+PIf4+r3pnzKdFe/ex94Ez0MMKKXKiEl3uh6qQLRYFIfdjzpDPuFw60eIzVrEnvfvYv2ew1anUkpZoLSHKJ89ezZvvPFGsXMXlN46W1wiRPV+jb8qR3HDz8+zbnIPfuw5hZva6kRKSlVUo0aNIiUlhQ8//DDf18ePHw/A3r176datW77De4BrXCiHI/+v6V69epVM2ALSI4sSEnHzcE53m0C0/EHt2b35aukqqyMppSxSnCHKr732Wp577jk6d+7Me++9x9y5c+nQoQNt27alS5cuJCQkADBx4kRGjhwJwMCBAxkxYgTXXHMNjRo1OjtcSEnSI4sSVCmmH2mVIqj3+b2ELhvAR8ff4/5et2Oz6TDnSpWa756GI5s8tzviHoivINctaraC28cUL1chnDp1iuXLlwNw4sQJYmNjERE++OAD3nrrLcaOHXvBNgkJCaxYsYJNmzbRr1+/Ej/y0COLEhbY9Cb8hn1PJT/ou3EY7346lfQsvbVWKVVw/fv3P/t8//79dOnShVatWvH222+zZcuWfLfp2bMnIkLr1q05ePBgiWfSIwsvcNSOJvSRHzj5USwP73uC/3v/GA8+8DiVg/ysjqZU+VfQI4CcI4qh33ovSxGFhIScff7oo4/y7LPPcscdd7BkyRLGjMl//wICAs4+98boEnpk4SVStSHhjy0lpcrl/CNxNB//34scTEq1OpZSysecPHmS2rVrY4xh8uTJluXQYuFNIRFUfXghp2pdyz9S3+Ob/z7BloNJVqdSSnlZcYYoz+vFF1+kV69eXH/99dSoUaMEUxaODlFeGrIzOTnz71Te+RWfmy7Uuee/XHd5TatTKVVuFGWI8rJ8GspbijNEuR5ZlAa7H5X7T+R0+0cYIIs4M+0+vl79h9WplKrYhn5boQpFcWmxKC02G6HdXyPtppfpaltNrfn38sHCdTrMuVLKJ2ixKGWBnYeT1fMjYmy7uH7FIF6duZTMbKfVsZRS6pK0WFjA0aYf9oFf0NiRyOBtD/D8x19zJj3L6lhKKXVRWiwsIpfdhP+wBVQLcDLq4AieHz+JBB3mXClVRmmxsFKttgQ99AOBYVV55eSzvPHf/2N3wmmrUylVIQz9fihDvx9qdQyf4dViISJdRWSHiOwWkacv0qafiGwVkS0iMj3X+mwRWe9+zPNmTktVbUTwQz9gIpryWsZrfPb+f1iz97jVqZRShZQzRPn69eu5+uqradGiBa1bt2bmzJkXtC2JIcoB1q1bx/fff18i+T3x2nAfImIHxgO3AvHAGhGZZ4zZmqtNE+AZoJMx5oSIVM/1FqnGmDbeylemhFYn+MHvSZ16D6MPvM9bHx8noc8L3Bldy+pkSqlCCg4O5rPPPqNJkyYcOnSI9u3bc9tttxEeHn62TUGHKPdk3bp1bN68ma5du5ZI9kvx5pHFVcBuY8weY0wGMAPokafNA8B4Y8wJAGNMghfzlG0BYQQN/oqMZr35p30Gx74YycTlu61OpZQqpKZNm9KkSRMAatWqRfXq1Tl27FiBt9+1axe33XYb7du3p3PnzuzcuROAGTNm0LJlS6Kjo7nxxhtJTU1l9OjRTJs2rUhHJYXlzYEEawMHci3HAx3ytGkKICIrADvwojEm55gqUETigCxgjDFmjhezlg0Of/z7fkzW9zUYsvp/zF98kldOjOGZ7m2w6zDnShXI2NVj2X58u8d2OW0Kct3iiqpX8NRVTxU6y+rVq8nIyKBx48YF3ubBBx9k4sSJNG7cmBUrVvDYY4+xaNEiXnrpJZYtW0aNGjVISkoiKCiI559/ns2bNzNu3LhCZyssbxaL/L7d8vZAcwBNgBuAOsDPItLSGJME1DPGHBKRRsCPIrLJGHNet2cReRB4EKBevXolnd8aNhuOO8bgrFSLbkv+H1XjHuFfSWN47Z5rCfSzW51OKVVAhw8f5r777mPy5MnYbAU7iZOUlMTKlSu56667zq7LynLdVt+pUycGDRpE37596d27t1cyX4o3i0U8UDfXch3gUD5tVhpjMoE/RWQHruKxxhhzCMAYs0dElgFtgfOKhTFmAjABXGNDeWMnrGK7djiEVafjnEcJ/+NxHv3wVd4c2oUqIf5WR1OqTCvoEUDOEcWkrpNKPMOpU6e48847eeWVV+jYsWOBtzPGEBERke81jI8++ohVq1Yxf/58oqOj2bhxY0lG9sib1yzWAE1EpKGI+AP9gbx3Nc0BbgQQkQhcp6X2iEgVEQnItb4TsJWKJro/tntm0tTvGC8eG8mI8V9w4HiK1amUUpeQkZFBr169zh4FFEaVKlWIioo6Oy2q0+lkw4YNAOzZs4eOHTvy8ssvU6VKFQ4ePEhYWBjJycklvg/58VqxMMZkAY8BC4FtwCxjzBYRGS0ise5mC4FEEdkKLAVGGWMSgWZAnIhscK8fk/suqgqlyS04/jafmoHZjEt5iufGT2ZjvA5zrlRZNWvWLJYvX86nn3569pbYwtztNGPGDD744AOio6Np0aIF8+fPB+CJJ56gVatWtGrViltuuYWWLVty0003sWHDBtq2bev1C9w6RLmvSPyDzMk9yTp1lOHZ/2DAvUO56QrrxrZXqiwpyhDl3jwNVVbpEOUVQbXG+D2wBEdkE/5nf535U95h+qr9VqdSymdN6jqpQhWK4tJi4UvCauB3/3dI/Wt42+9/7J33Km9+v/3cMOeT7jw3oYtSSpUgLRa+JrAS9vu+wtm8F8/6fU74Ly/yr5m/k5HlZMvhk2w5fNLqhEqpckiLhS9yBGDr8wnmqr8zzPEdnbc8x7BJKzhjAqxOppQqp7RY+CqbDbl9LNz8Aj3sv/Lggad54Uwf/nKGWZ1MKVUOabHwZSJw3T+gx/t0sm/jTcf/ePVMD5JSMqxOplSZt+++Qey7b5DVMXyGFovyoO29yIAZXG6L50O/t5mz0vO4OEqpklXaQ5TPnj2bN954o8Tye+LN4T5UaWrahXhHPRpk/cmJlVNw3vA6Nh18UKlSV5JDlGdlZeFw5P813atXr5IPfwl6ZFGOnJEQ/iKcW9MW8cvuv6yOo1SFVNwhyq+99lqee+45OnfuzHvvvcfcuXPp0KEDbdu2pUuXLiQkuGZymDhxIiNHjgRg4MCBjBgxgmuuuYZGjRqdHS6kJOmRBeWoJ6cIabYQWrKXV35aTOem91idSKlSd+TVV0nf5vlUbNp2V5uCXLcIaHYFNZ99ttBZijJEObgGIly+fDkAJ06cIDY2FhHhgw8+4K233mLs2LEXbJOQkMCKFSvYtGkT/fr1K/EjDy0W5UiLqMrgDCHz4F802v8lh5J6USs8yOpYSlVIRRmiPEf//v3PPt+/fz/9+vXjyJEjpKen07Rp03y36dmzJyJC69atOXjwYLGy56fCFwuncdJk7gYuP2oH789M6H02BxlNY+m+bT6TftvO8NvbWp1IqVJV0COAnCOK+lM+K/EMRR2iPEdISMjZ548++ijPPvssd9xxB0uWLGHMmDH5bhMQcK6flTfG/Kvw1yz2H9tNqjOdXxuWn9tNQ66+nzBJJSluFhlZTqvjKFWhFGeI8vycPHmS2rVrY4xh8uTJJZCwaCp8saidFUq/n5003ZvJjuM7rI5TPEO/dT3qdeRMpcZ0y1zEoq1HrE6lVIVS3CHK83rxxRfp1asX119/PTVqWDfStA5RDvzcqTURiZlM+Vdr/nP/DER8/5ZT54r/Ylv8b/4V+QFvPjrA6jhKeVVRhij35mmoskqHKC+mqaOiOR1io92sjfy47wer45QIW5t7yBYHLQ7PZtfR0plJSylfUn/KZxWqUBSXFgvgw55TaPiPZ2ixH76b+jLp2elWRyq+kGpkNb2TXvZfmPHrTqvTKKV8nBYLt2r97ia7bk26Lkhg6sZPrY5TIgKuGkq4nOH0+jmcSc+yOo5SXlVeTql7S3H/fLRYuImfH/WfeZ7ax2H3Zx9wLKXgPS7LrIbXkx5al57OJcxdf8jqNEp5TWBgIImJiVowLsIYQ2JiIoGBgUV+jwrfzyK30BtvwNa+NT1/2sj7v7zJC10u7CXpU2w2/K8awtU/vszHK35lwFV1y8XFe6XyqlOnDvHx8YUaVqOiCQwMpE6dOkXeXotFLiJCvWeex9mnD36ff8PmdvfRMqKl1bGKRdoOxLn0VWKOz2fd/ttoX7+K1ZGUKnF+fn40bNjQ6hjlmp6GyiOoZQuCu91BtzWG/y0c7fuHtWE1cV7Whb725cz4bbfVaZRSPkqLRT5q/eOf2G0Oor/ezHd/fmd1nGJzXDmUanKS1M3fcvxM+emprpQqPVos8uFXqxYRg4dw3RbDF/PGkJKZYnWk4rnsFjJDougrPzAr7oDVaZRSPkiLxUVE/P3vmPBK3LngGJ9u9vGhy212/GIGcZ19E0t+W4PT6eOn1pRSpU6LxUXYQ0OJGjGSFvvh99kfcfj0YasjFU/bgQhw3emF/LRL7xhRShWOFotLCO/TB1uDegxYksG41W9ZHad4wuthGt3E3Y6fmP7bHqvTKKV8jBaLSxA/P2o9+TRRxw3Zc75j3dF1VkcqFlvMYGqSSPauJRw47uPXYZRSpUqLhQehN95AwFUx9PsF3vnpPziND88P0fR2soMj6G/7kc9X77c6jVLKh2ix8EBEiHrqacJSnDRfsI25u+daHanoHP7Y297Lzfbf+WHNRtKzsq1OpJTyEV4tFiLSVUR2iMhuEXn6Im36ichWEdkiItNzrR8sIrvcj8HezOlJUIsWVIqNpVscTPnxbU5nnLYyTvG0G4wdJzenLeb7zToxklKqYLxWLETEDowHbgeaAwNEpHmeNk2AZ4BOxpgWwEj3+qrAC0AH4CrgBRGxdJyK6k+MxG5z0HVRIhM2TbAySvFUa4ypfy0D/X9i+m97LQ6jlPIV3jyyuArYbYzZY4zJAGYAPfK0eQAYb4w5AWCMSXCvvw1YbIw57n5tMdDVi1k98ouKImLo37hui2HFks84cMp3O7dJ+yHUMkexH/iF7UdOWR1HKeUDvFksagO5v1Hj3etyawo0FZEVIrJSRLoWYttSV+2BYUiVcAb+kMWba96wOk7RNeuOMzCcex1LmbZSL3QrpTzzZrHIbyzsvF2HHUAT4AZgADBRRMILuC0i8qCIxIlIXGkMTWwPDaXG8OFcsT+bUz/+wMrDK73+mV7hF4gtuj+32eP4cd1WTuvESEopD7xZLOKBurmW6wB5Z+CJB+YaYzKNMX8CO3AVj4JsizFmgjEmxhgTExkZWaLhLya8b1/8GjVkyE823vhtDFlOH/2ibTcYh8mka/YyZv9+0Oo0SqkyzpvFYg3QREQaiog/0B+Yl6fNHOBGABGJwHVaag+wEOgiIlXcF7a7uNdZThwOaowaRfW/smi4dBdf7vzS6khFU6M5ps6VDAn4iWm/7fX9odiVUl7ltWJhjMkCHsP1Jb8NmGWM2SIio0Uk1t1sIZAoIluBpcAoY0yiMeY48DKugrMGGO1eVyaE3nADwR06MOBXG5+s/C8n009aHalIpN1g6jrjCU2II27fCavjKKXKMCkvv1HGxMSYuLi4Uvu81C1b2NunL3M7Cjx8H09flW83krIt4wzmzabMy2jHj1eM5t3+ba1OpJQqZSKy1hgT46md9uAuoqAWLagcG0u3NbBk5ef8kfSH1ZEKzz8EadWXO2QVP2/azV+n061OpJQqo7RYFEPkyBHY7Q7uXQ5vrHnDN8/7tx+Mn0nnDlYwc43v9h1RSnmXFoti8IuKotqQoXTcnMGRuF/4+eDPVkcqvFptoWZrhgUtZ/rKfWTrxEhKqXxosSimag8Mw16tKg8s9+P11WPJzM60OlLhtRtEg6w/qHpqK8t2JHhur5SqcLRYFJM9NJTIxx+n0Z9pVF+7l+nbp3veqKxp3Q/jCGJo0E9MXbnP6jRKqTJIi0UJCO/TB/9GjRi2PICP1v2PxNREqyMVTmBlpEUvurGC1TsP6MRISqkLFLpYiIhNRCp5I4yvEoeD6qP+RZVjqXRac4b31r9ndaTCaz8Yf2cK3e0rmbZKx4tSSp2vQMVCRKaLSCURCQG2AjtEZJR3o/mW0BtuILhjRwb8aue7jV+y/fh2qyMVTt0OEHE5D4b8zKy4A6Rl6sRISqlzCnpk0dwYcwroCSwA6gH3eS2VDxIRajw5ioAzGdy92o+xq8f61q20ItB+MI3StxGZspvvNh+2OpFSqgwpaLHwExE/XMVirjEmk3xGga3oAps3p3JsLF1WZ7JvxxoW71tsdaTCad0fY/fnwdBfmKpDlyulcilosfgQ2AuEAMtFpD6gs+bkI3LkCOw2Ow/8FsLba98mLSvN6kgFF1INadadO81yNu87ytZD+leslHIpULEwxvyfMaa2MeYO47IP92ix6nx+UVFUHTqENutPEbgzns+2fmZ1pMJpN5jArFN094tj6iq9jVYp5VLQC9wj3Be4RUQ+FpF1wE1ezuazqg17AHu1agxfUYmJGz8iIcWHOro1uA6qNODhsF+Y8/tBTqX5YCdDpVSJK+hpqL+5L3B3ASKBocAYr6XycfbQECIff4yo3SeI3p7Bu+vetTpSwdls0G4QjVPWUyMzntnrdGIkpVTBi0XONKd3AJOMMRvIf+pT5Rbepw/+jRvz91+C+HbnXDYe22h1pIJrcy+IncfDf2Xqyn2+dVeXUsorClos1orIIlzFYqGIhAFO78XyfTkd9UKOnKTX5mDGrh6L0/jIH1lYTbj8du7IXsrehCRW/Vlm5p1SSlmkoMXifuBp4EpjTArgj+tUlLqE0OuvJ7hjR3r/ks3u+A18u+dbqyMVXLvBBGYcJzZwvY4XpZQq8N1QTqAO8G8ReRO4xhjjQ+dVrJHTUc+RnMoDv1dl3NpxpGT6yLhLl90MlerwcKUVfL/5CAnJPnQLsFKqxBX0bqgxwAhcQ31sBYaLyGveDFZeBDZvTuUePbhmxQnM4aN8vPljqyMVjM0ObQfS+NRqapoEZunESEpVaAU9DXUHcKsx5hNjzCdAV+BO78UqXyJHjsBms/PPtTWZvGUyB0/7yB1GbQciwD8jVjN91X6ysn3kmotSqsQVZtTZ8FzPK5d0kPLMr2ZNqg4dQqPVB2l80MnbcW9bHalgwuvCZbdwe9YSjp48w4/bfai/iFKqRBW0WLwG/C4in4rIZGAt8Kr3YpU/OR31Rv5ahUV7FxJ3JM7qSAXTbhCBqUfpGbqNqTp0uVIVVkEvcH8OdAS+dj+uNsbM8Gaw8sbVUe9xwnccosv+cMauGUu20weGAb/8dgipziOVfmH5zmPs/euM1YmUUha4ZLEQkXY5DyAKiAcOALXc61QhhPe5C//GjRm8TNh1bBtzds+xOpJndj9ocw+NTqwgynaC6av16EKpisjTkcVbl3i86d1o5U9ORz2/Q38xdFdt/u/3/yM5I9nqWJ61G4SYbJ6JWqsTIylVQV2yWBhjbrzEQwcSLILQ668n+OqO3PrjCdKTjjNh4wSrI3lWrTE0uI4u6Ys5mZLO/I06MZJSFU1B+1n0zudxs4hU93bA8sbVUe9J5NRpntzWmKnbprLvlA/0kG4/hMDTB+hT5Q/t0a1UBVSY4T4mAve6Hx8B/wBWiIhOr1pIgc2aUblHD5r9sIdayX68ucYHzuhd0Q2CqvBIpRWsP5DE5oMnrU6klCpFBS0WTqCZMeYuY8xdQHMgHegAPOWtcOVZ5MgRiN3O02trsyx+Gb8e/NXqSJfmFwjRA2hwbCm1/M7o0YVSFUxBi0UDY8zRXMsJQFNjzHFAZ8cpgpyOehErttMpKZLX17xOljPL6liX1m4Q4szkudrrmbP+ICdT9a9eqYqioMXiZxGZLyKDRWQwMA/XXNwhQNLFNhKRriKyQ0R2i8jT+bw+RESOich692NYrteyc62fV9gd8wXV7h+GvVo1Hv45mD+SdjNrxyyrI11a9WZQ5ypuSf2etMxsvl4Xb3UipVQpKWixeBSYBLQB2gKTgUeNMWeMMfnOxS0idmA8cDuu01YDRKR5Pk1nGmPauB8Tc61PzbU+tqA75EtyOur5b/6De481Yfz68SSlXbT2lg3tBxNw8g/uqXGQKToxklIVRkF7cBvgF+BHYAmw3Hj+lrgK2G2M2WOMyQBmAD2KE7Y8Cu9zF/6XNabHwpOkpiXz/ob3rY50aS16QUAl/h72C3uOneG3PxKtTqSUKgUFvXW2H7Aa6AP0A1aJSB8Pm9XG1ds7R7x7XV53ichGEflSROrmWh8oInEislJEehYkpy8Sh4Mao0ZB/GFGxbdm1o5Z7D6x2+pYF+cfAq36UO/IIuoGpTN1lV7oVqoiKOhpqOdwzZI32BgzCNdRw//zsE1+c3TnPRr5BtfF89a4jlgm53qtnjEmBrgHGCcijS/4AJEH3QUl7tixYwXclbInpHNngq/uSNv5u4jIDuL1Na+X7dM77QYjWWn8u94WFm45ytFTOjGSUuVdQYuFzRiTe3zqxAJsGw/kPlKoAxzK3cAYk2iMSXcvfgS0z/XaIffPPcAyXNdKyLP9BGNMjDEmJjIysoC7UvbkdNQzp5L59/Yr+O3wbyw7sMzqWBdXqw1ERXPjmQVkO53MWK0TIylV3hW0WHwvIgvddy8NAb4FFnjYZg3QREQaiog/0B/XXVRniUhUrsVYYJt7fRURCXA/jwA64Zqhr9wKbNaMyj17UmPBWtpn1+XNuDfJyM6wOtbFtRuM/19bGVz/BNNX7yNTJ0ZSqlwr6AXuUcAEoDUQDUwwxlyyM54xJgt4DFiIqwjMMsZsEZHRIpJzd9NwEdkiIhuA4cAQ9/pmQJx7/VJgjDGmXBcLgMgRwxG7nZFrItkv3eoLAAAcq0lEQVSfvJ9p26ZZHeniWvUFv2AeCFnO0VPp/LDtqOdtlFI+S8r0ufFCiImJMXFxPjKh0CUkvPsuif/7gC//FcO3wbuY32s+EUERVsfK35xHMFvncgsfElU9kqnDOlidSClVSCKy1n19+JI8zWeRLCKn8nkki8ipkourclS7fxj2iAj6LUohPSuN//7+X6sjXVy7wUjGaZ6rv41fdv/FnmOnrU6klPIST0OUhxljKuXzCDPGVCqtkBVJTkc958at/PP0tczeNZutiWX0DFzdqyDyCq5L/haHTZim064qVW4V9AK3KkXhd/XG/7LGdJizkwhHOGNXjy2bt9KKQLvB+B35nb81OcMXcQdIzdCJkZQqj7RYlEE5HfWy9h/g34evZF3COhbuW2h1rPxF9we7P0ODfuZUWhbfbDzkeRullM/RYlFGhXTuTMg1V1P7ixVEB1zG23Fvk5ZVBju/BVeFZrHU3DuHFpF+OnS5UuWUFosySkSoPmoUzlOnGLWtIYfPHObTLZ9aHSt/7QcjaSd5usFONsafZMOBMj4YolKq0LRYlGE5HfUCvl5C75BOfLL5E46cOWJ1rAs1uA6qNuLqpPkE+9v16EKpckiLRRkXOXIE2O0M/EnIdmYzbt04qyNdSATaDcJx4DeGNctm3oZDJKWU4d7nSqlC02JRxvnVqEG1vw0la/Eyhvvfxrd7vmV9wvp82w79fihDvx9aygndou8Bm4NBAT+RnuXky7U6MZJS5YkWCx9Q9W/3Y4+I4Lo5e4gMjGDs6rE4TRkbiymsBjTtSsTur+hQL5Rpq/bjdJbB232VUkWixcIH5HTUS/99A/9Ov5XNiZuZv2e+1bEu1H4IpPzFP+vt5s+/zrDij7+sTqSUKiFaLHxE+F29CWhyGfWmLiM6vAXj1o4jJTPF6ljna3wTVK5LzPFvqBrirxe6lSpHtFj4CHE4qD5qFJn7D/DUwWiOpR5j4qaJnjcsTTY7tB2Ibc8yHmhlZ/HWoxw+mWp1KqVUCdBi4UNCrruOkGuuJmDyXHrX6MLkLZOJTy5jF5LbDgQR7vH7CQN8rhMjKVUuaLHwISJC9SefJPvUKYbEhWK32Xl77dtWxzpf5Tpw2S1U3j6Tm5pUZcbq/ToxklLlgBYLHxN4xRVU7tWL9JmzeTSyD4v3LWbNkTVWxzpfu8GQfJjh9faSkJzO4q06MZJSvk6LhQ+KHDEc7HZuXHCQWiG1GLN6DNnOMjTaa9PbIKQ6rRPmUDs8iCm/6YVupXydFgsflNNR78z3i3g6pDc7T+zkq11fWR3rHLsftL0X2bWIB9sG8tueRHYnJFudSilVDFosfFROR72Gn/1E++rteO/398hyZlkd65x2g8A46WP7CT+7MHWlToyklC/TYuGj7KEhRA5/nNTff+epM9eTlJ7E4TOHrY51TtVG0LAzIVumc2fLGny1Np6UjDJUzJRShaLFwoeF93Z11PObMJM+DXuQkJJQtua8aDcYkvbzcP14ktOzmLdeJ0ZSyldpsfBh5zrq7WfozlrYxMb+5P1lZwrWZt0hqCpN47/mipphfPbbvrKTTSlVKFosfJyro941pHw0mZ6r4VTGKYZ8P4Qdx3dYHQ0cARA9ANn+Lfe3C2Pr4VP8rhMjKeWTtFj4OFdHPdeMel3XOOkT58efJ/+k3/x+vLLyFZLSLP5ybj8YnJnE8hMhOjGSUj5Li0U5kNNRr9LJTDrtsPFNr2/of3l/vtz5Jd3mdGPm9pnW9cOIvBzqdiRgw1R6t63N/I2HOXFGJ0byVR0m3UWHSXdZHaPYyst+QOntixaLciJyxHCMQHhiOpUDKvNMh2eY1X0WTas05ZVVr3D3/LtZe3StNeHaDYLEXTxQ/wgZWU6+WFuxxosqT19MquLSYlFO+NWoQXK4PyFnsjj84os4z5yhaZWmfNzlY968/k1OZpxkyPdDeHL5kxw9U8rDb7ToCQGVqLf3C65qUJWpK3ViJKV8jRaLcuRkFX9OVfYjaeYs9vToScqaNYgItzW4jbk95vL31n/nh30/0H1OdyZumkhGdimdDvIPgVZ9YetchrSrzP7jKSzfdeySm9z94W/c/eFvpZNPKeWRFovyRISkiEDqT50CIuwbNJijr43BmZZGsF8wj7V9jDk953B11NW8u+5des7tyU8HfiqdbO0HQ1Yat2YvJyLU32OP7r3+b7LX/83SyaaU8kiLRTkU3L49jebMpsqA/hyfPJk/e/UmdcMGAOqG1eXdm97lw1s+xC52HvvxMR5Z8gh7T+71bqioaIhqg9/6KdwdU4cftx/lYJJOjKSUr/BqsRCRriKyQ0R2i8jT+bw+RESOich692NYrtcGi8gu92OwN3OWR7aQEGo+/zz1PvkYZ1oaewfcQ8I743BmuE49XVP7Gr6O/Zp/xfyLdQnr6DWvF++sfYczmWe8F6r9YDi6mUH1j7smRlql40Up5Su8VixExA6MB24HmgMDRKR5Pk1nGmPauB8T3dtWBV4AOgBXAS+ISBVvZS0vZjzeghmPtzhvXcg119Bo3lwq9+xJ4ocfsrdvP9K2bwfAz+7H4BaDmd9rPnc2vJNPNn9C99ndmb9nvnd6WrfsA37B1Ng1g5uvqM6MNfvJyNKJkZTyBd48srgK2G2M2WOMyQBmAD0KuO1twGJjzHFjzAlgMdDVSznLPXtYGLVe/Q91/vc+WccT+bNvP/764ANMlmtgv4igCF659hWm3jGV6sHVeebnZxj8/WC2JW4r2SCBlaBlb9j0FYPaV+Ov0xks3HKkZD9DKeUV3iwWtYHcN9THu9fldZeIbBSRL0WkbiG3VYUQduONNJo3j0q33sqxce+yd8A9pP/xx9nXoyOjmX7ndF665iX2ndrH3fPvZvRvozmRdqLkQrQbAplnuDZtOXWrBjFFe3Qr5RO8WSwkn3V5z218AzQwxrQGlgCTC7EtIvKgiMSJSNyxY5e+FVO5OKpUofbbb1H7nbfJPHCAP3v1JvGTSZhsVw9vm9jo3aQ33/T6hnub3cvXu76m2+xufL7985KZL6NODEQ2w7ZuMgM71Gf1n8fZcUQnRlKqrPNmsYgH6uZargOcN0a1MSbRGJPuXvwIaF/Qbd3bTzDGxBhjYiIjI0sseEVQ6fbbafTNPEKuvZaE119n36DBZOw/d8G5kn8lnrrqKb7s/iXNqjbj1VWvcvf8u4s/37eI60L3oXX0r3cSf4eNaav06EKpss6bxWIN0EREGoqIP9AfmJe7gYhE5VqMBXJOki8EuohIFfeF7S7udaoEOSIjqTP+PaLGvEb6zp3s6dGTE59/ft7F7cuqXMZHXT7i7RveJjkjmb8t/BujfhrFkTPFuNbQ+m6wB1B52+d0axXF1+sOciZdJ0ZSqizzWrEwxmQBj+H6kt8GzDLGbBGR0SIS6242XES2iMgGYDgwxL3tceBlXAVnDTDavU5dwqSuk5jUdVKhthERwnv2pNG8uQS3bcuRl0Zz4P5hZB4+fF6bW+vfytyec3k4+mGWHlhK7JxYJmycQHp2+iXe/SKCq0LzWNg4k4ExNTidnsWc9QcL/z5KqVLj1X4WxpgFxpimxpjGxpj/uNc9b4yZ537+jDGmhTEm2hhzozFme65tPzHGXOZ+FO4bUBWaX1QUdT+eSM0XXyRl/Xr2dI8l6evZ5x1lBDmCeKTNI8ztOZdOtTrx39//S885PVm6f2nhb7VtNxjSTtI2eRnNoyoxRSdGUqpM0x7c6iwRoUr/u2k0dw4BV1zO4WefJf6RR8nKc/NA7dDavHPjO0y4dQL+dn+GLx3Owz88zJ8n/yz4hzW4Fqo2RtZ9xsCO9dl+JJm1+0rwriulVInSYqEu4F+3LvU/+4zqTz/FmV9/ZU+37pxasOCCdlfXupovY7/kySufZEPCBnrP7c1bcW9xOuO05w8RcQ1dvv9XetY9TViAQydGUqoM02Kh8iU2G9WGDKHh7K/xq1+fg//4J/FPPEHWifN/+/ez+XFf8/v4ptc3dG/cnU+3fEr3Od355o9vcBoPvbPb3AM2B8GbpnFX+zos2HSExNNFuAailPI6LRbqkgIaNaLB9GlEjhxJ8pIf2NM9luQff7ygXURQBKM7jWb6HdOJConi2V+eZdB3g9iSuOXibx5aHS6/HTZ8zsCYGmRkO5kVF+/FvVFKFZUWC+WROBxEPPR3Gn4xC0dEBPGPPMqhZ54lO/nCznStIlsx9Y6pvNzpZQ4kH2DA/AG8+OuLHE+7yM1s7YZASiKXnfiZjo2qMm3VPrJ1YiSlyhwtFqrAAq+4goazZlLt4Yc4OW8ee7rHcvqXFRe0s4mNnpf1ZH6v+dzX/D7m7p5Lt9ndmLZt2oW9wBvfCJXrwtrJDOxYn/gTqSzfqb3xlSprtFioQhF/f6qPGEGDz6djCw7mwLBhZ6dxzSvMP4xRV47iq9ivaFGtBWNWj6HvN31ZfXj1uUY2O7S9D/YspUtUOpFhATpelFJlkBYLVSRBrVvT8OuvqDp0qGsa1569SFmT/1AgjcIbMeHWCYy7YRypWancv+h+/rnsnxw+7e7413YgiA3/jVMZcGVdlu5IIDsjtBT3RinliRYLVWS2wEBqPPUk9ad8BuCaxnXMWJxpaRe0FRFurn8zc3rM4dE2j7I8fjmxc2L5YMMHpIdGwGW3wvpp9I+phQApJy8v5b1RSl2KFgtVbMExMTSaM5vw/ndz/NNP+bP3XaRu3Jhv20BHIA9FP8TcnnPpXKcz49ePp8ecHvzQMAaTfJhaCT9zS7MapJ5sinHqP0+lygr9v1GVCFtICFEvvEDdjyfiTElxTeM6bhzGPY1rXrVCa/HWDW8xsctEghxBjNw1hYdq12HP2gkM7Fgfkx1E2ukGpbsTSqmLclgdQJUvoZ060WjeXI6+NobEDz7k9NJl1Bo7hsArrsi3fYeoDnzR/Qtm7pjJ+DVvcVfmbu5JeA+7fwOSj15N7Hu/EOhnJyjn4W8/t+xvI8jPvex/rk2g//nt87ax2/KbLkUpdSlaLFSJs1eqRK3XXiXs1ls5/Pzz/Nm3H5GPPkK1YcMQx4X/5Bw2B/c2u5fbw5vzf1/1ZsofXxPa0I5Jbk5GSFPOZENWJmSlQWa2ITNbyMwyZGQbjBGMEUDA2Fw/L3guGASwgREcNjv+dhuBDj8C/Oz42x0EOOwE+NkJdDgIcDgIdDgI9HOcXQ7ycxDk5+da5+cg2L3sKkB+BPs5CA5wEOznT5C/gyCHA7vNjk1sGPe8XZnZmZic/8y5nwAGg9M4z19nTL7tz7Z1GrKN62GME6cxON3LTmc22QacOeudxv266zOyjSHb6cTpdGI41871Xq52Tmeu7d0/09P9AZi1abk794VMnp77edt4GjDSWYABJfO2yfue+ec69zwtPRAwfLJ2kcfPKjYvdxtKSwtEbN7vmyTlZaTPmJgYExcXZ3UMlUfWiRMcffllTi34jsBWrag1dgwBjRpdfIPJsWw5+SdDwyBVT5IqVSDOtJps+fviIm0rImuNMTGe2umRhfIq1zSubxN2yy0ceWk0f/bqTeTIkVQddB9it1+4QfvBtPjyb8RSg/WBQUz520qMMWSbbNdvxe7nOb9d536cbUeudiZPO5wXbJff++beNsuZTUZWNunZWaRlZpGelU16Vhbp2a7nGVnZZGRnnf2Znp1NZlY2mdmu5c3HdgNC06oNsYlw9j/JeYBgw5Z7HeJqK+fa5t3WJjZsQq73smED1085fxvbee/ret0mAu6fNvd61yrX+9okJ9O55Y9+nwHAQ+0HnP0rk/xmQZbz1+VtIYV8Pb/P8fweF8bKaTVu1ScA/KPj/fk1KnHePPH51sqPcYj3f+nXYqFKRaU77iD4yis5/PwLJIwdS/IPS6j16qv416t3fsMrukFQVW5OSWZjYDBBjiBrApegDpPuAmDOvS9YnKT4pmx3fck+3OEOi5MUz/82vA/AoLY3WZyk+N5bP75UPkcP9FWpcURGUuf98US99hrp23ewp2cvTsyYcf75ZkcAtLmHDikpPDg71bqwSqnzaLFQpUpECO/Vk0bfzCO4TRuOvPjSBdO40m4QNhvUrp5iXVCl1Hn0AreyjDGGpJkzOfr6G4jNRo3nnqNyzx6ICMefiKRq5QwIjoBKtc5/hOU8rw2VoiAgzOpduaQFtzQH4I4lWy1OUnzlZV/Ky35A8fdFL3CrMs81jWt/Qq65hkPPPsvhZ54hedEioka/xKbd4URFpNK0d3c4dQhOHYT4NZCSeOEbBVRyF5EodwGp5SoiZ5/XhqAqF7viqZQqAC0WynL+9epRf/Jkjn82hWPvvMOebt3B7mB3aiWadh93fuPMNEg+BKcOnysiyYddP08dgj92wOkjkHeWPkdgrmISda6I5C4wodVdo+AqpS6gxUKVCWK3U23oEEI7X8ehp5+h+qZNpCUbDjz0MLbgICQ4GFtQMLagIGzBwa51QUHYgptgC4rGVt+1zhYUhC0wAHGexpaVhKQlIGeLibvAHFjtKjDZGXlDQFjNfI5Scp8Ci3JdhFeqgtFiocqUgMaNafD5dH69phXBaZB59CgmJQVnaipO90+yswv+hjYbtqAgJNhdZIKCsQW1xBZ8FTZ/O+JwYrM7sUkGNtKwJaW4C80mbFnLsZGKzW6w+TkRu8HmZ7CFVcVWNQoJr3Pxo5QAHWJdlS9aLFSZIw4HJysJJytB29lfn/eaMQaTmYnzzBlMauq5IpKSijM1JVdhySkuKa52Z9u4nmefTiYr5VwBcqamYlLz3qob7H7kJxGxJ2JzrEfs2dgcxvVwFxTxd7iOckJCaWpLxQDHHo/Nc91E8vTWkvN+XNguzws575X3Wkzezmpnl+UibS7yPjmfK+eWG4lr+Pm/nh2UN6RPKS/7AdBQ0sgqhfHOtFgonyIiiL8/Nn9/qFKlRN/bOJ3nF6CzRSbl3PozOcXFXZhSUnGeScZ56jjO0ycxp5PJOnPGtf54Bs70JJwZAWDgr/07c+9JiWYvPa5TcMcO5D/Rle8oL/sBEEBQeKbXP0WLhSqT0qX0e26LzYaEhGALCSnR9y3urY353t5eUutynjudF7RxDYBozmu3KPZKALrMXeUhddm2qEcHALrMWWlxkuJb1LMj4MdlXv4cLRZKlXH5jZNUGrcB5/cJTvdaW0glr3++N53dj9DKFicpPmcpHaVqD26llFIe6ZGFKpPG3NMAgF7WxlBKuemRhVJKKY+0WCillPLIq6ehRKQr8C5gByYaY8ZcpF0f4AvgSmNMnIg0ALYBO9xNVhpjHvJmVqW85aV7mwHg2zNAuJSXfSkv+wGlty9eKxYiYgfGA7cC8cAaEZlnjNmap10YMBzIey/eH8aYNt7Kp5RSquC8eRrqKmC3MWaPMSYDmAH0yKfdy8DrQJoXsyillCoGbxaL2sCBXMvx7nVniUhboK4xZn4+2zcUkd9F5CcRuS6/DxCRB0UkTkTijh07VmLBlVJKnc+bxSK/niJnu46KiA14B/hnPu0OA/WMMW2BfwDTReSCXkDGmAnGmBhjTExkZGQJxVZKKZWXNy9wxwN1cy3XAQ7lWg4DWgLL3D1UawLzRCTWGBMHpAMYY9aKyB9AU0Cnwqsgmkf5dg9hpcobbx5ZrAGaiEhDEfEH+gPzcl40xpw0xkQYYxoYYxoAK4FY991Qke4L5IhII6AJsMeLWZVSSl2C144sjDFZIvIYsBDXrbOfGGO2iMhoIM4YM+8Sm3cGRotIFpANPGSMOe6trKrsmdR1ktURlFK5eLWfhTFmAbAgz7rnL9L2hlzPvwK+8mY2pZRSBac9uJVSSnmkxUIppZRHku/EKj4oJibGxMXpzVJKKVUYIrLWGBPjqZ0eWSillPJIi4VSSimPtFgopZTySIuFUkopj7RYKKWU8kiLhVJKKY+0WCillPJIi4VSSimPtFgopZTyqNz04BaRY8C+YrxFBPBXCcWxUnnZD9B9KavKy76Ul/2A4u1LfWOMx9njyk2xKC4RiStIl/eyrrzsB+i+lFXlZV/Ky35A6eyLnoZSSinlkRYLpZRSHmmxOGeC1QFKSHnZD9B9KavKy76Ul/2AUtgXvWahlFLKIz2yUEop5ZEWCzcReVlENorIehFZJCK1rM5UVCLyhohsd+/PbBEJtzpTUYlIXxHZIiJOEfG5O1dEpKuI7BCR3SLytNV5ikNEPhGRBBHZbHWW4hCRuiKyVES2uf9tjbA6U1GJSKCIrBaRDe59eclrn6WnoVxEpJIx5pT7+XCguTHmIYtjFYmIdAF+NMZkichYAGPMUxbHKhIRaQY4gQ+BfxljfGY6RBGxAzuBW4F4YA0wwBiz1dJgRSQinYHTwGfGmJZW5ykqEYkCoowx60QkDFgL9PTFvxcRESDEGHNaRPyAX4ARxpiVJf1ZemThllMo3EIAn62ixphFxpgs9+JKoI6VeYrDGLPNGLPD6hxFdBWw2xizxxiTAcwAelicqciMMcuB41bnKC5jzGFjzDr382RgG1Db2lRFY1xOuxf93A+vfHdpschFRP4jIgeAe4Hnrc5TQv4GfGd1iAqqNnAg13I8PvqlVF6JSAOgLbDK2iRFJyJ2EVkPJACLjTFe2ZcKVSxEZImIbM7n0QPAGPOcMaYuMA14zNq0l+ZpX9xtngOycO1PmVWQffFRks86nz1iLW9EJBT4ChiZ58yCTzHGZBtj2uA6g3CViHjlFKHDG29aVhljbilg0+nAt8ALXoxTLJ72RUQGA92Am00ZvzBViL8XXxMP1M21XAc4ZFEWlYv7/P5XwDRjzNdW5ykJxpgkEVkGdAVK/CaECnVkcSki0iTXYiyw3aosxSUiXYGngFhjTIrVeSqwNUATEWkoIv5Af2CexZkqPPdF4Y+BbcaYt63OUxwiEplzt6OIBAG34KXvLr0byk1EvgIux3XnzT7gIWPMQWtTFY2I7AYCgET3qpU+fGdXL+C/QCSQBKw3xtxmbaqCE5E7gHGAHfjEGPMfiyMVmYh8DtyAa4TTo8ALxpiPLQ1VBCJyLfAzsAnX/+8AzxpjFliXqmhEpDUwGde/Lxswyxgz2iufpcVCKaWUJ3oaSimllEdaLJRSSnmkxUIppZRHWiyUUkp5pMVCKaWUR1oslCoEETntudUlt/9SRBq5n4eKyIci8od7xNDlItJBRPzdzytUp1lVtmmxUKqUiEgLwG6M2eNeNRHXwHxNjDEtgCFAhHvQwR+Auy0JqlQ+tFgoVQTi8oZ7DKtNInK3e71NRN53HynMF5EFItLHvdm9wFx3u8ZAB+DfxhgngHt02m/dbee42ytVJuhhrlJF0xtoA0Tj6tG8RkSWA52ABkAroDqu4a8/cW/TCfjc/bwFrt7o2Rd5/83AlV5JrlQR6JGFUkVzLfC5e8TPo8BPuL7crwW+MMY4jTFHgKW5tokCjhXkzd1FJMM9OY9SltNioVTR5Df8+KXWA6QCge7nW4BoEbnU/4MBQFoRsilV4rRYKFU0y4G73RPPRAKdgdW4prW8y33togaugfdybAMuAzDG/AHEAS+5R0FFRJrkzOEhItWAY8aYzNLaIaUuRYuFUkUzG9gIbAB+BJ50n3b6Ctc8FptxzRu+Cjjp3uZbzi8ew4CawG4R2QR8xLn5Lm4EfG4UVFV+6aizSpUwEQk1xpx2Hx2sBjoZY4645xtY6l6+2IXtnPf4GnjGh+cfV+WM3g2lVMmb756Qxh942X3EgTEmVURewDUP9/6LbeyeKGmOFgpVluiRhVJKKY/0moVSSimPtFgopZTySIuFUkopj7RYKKWU8kiLhVJKKY+0WCillPLo/wMQ+lp8g2Ft+wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#比较训练集和测试集上的误差曲线（特征数存放于cv_results）\n",
    "test_means = -grid.cv_results_[ 'mean_test_score' ]\n",
    "test_stds = grid.cv_results_[ 'std_test_score' ]\n",
    "train_means = -grid.cv_results_[ 'mean_train_score' ]\n",
    "train_stds = grid.cv_results_[ 'std_train_score' ]\n",
    "\n",
    "#将penalty，C分解作为坐标轴行和列\n",
    "n_Cs = len(Cs)\n",
    "number_penaltys = len(penaltys)\n",
    "test_scores =  np.array(test_means).reshape(n_Cs,number_penaltys)\n",
    "train_scores = np.array(train_means).reshape(n_Cs,number_penaltys)\n",
    "test_stds = np.array(test_stds).reshape(n_Cs,number_penaltys)\n",
    "train_stds = np.array(train_stds).reshape(n_Cs,number_penaltys)\n",
    "\n",
    "x_axis = np.log10(Cs)\n",
    "for i, value in enumerate(penaltys):\n",
    "    #pyplot.plot(log(Cs), test_scores[i], label= 'penalty:'   + str(value))\n",
    "    pyplot.errorbar(x_axis, test_scores[:,i], yerr=test_stds[:,i] ,label = penaltys[i] +' Test')\n",
    "    pyplot.errorbar(x_axis, train_scores[:,i], yerr=train_stds[:,i] ,label = penaltys[i] +' Train')\n",
    "    \n",
    "pyplot.legend()\n",
    "pyplot.xlabel( 'log(C)' )                                                                                                      \n",
    "pyplot.ylabel( 'logloss' )\n",
    "pyplot.savefig('LogisticGridSearchCV_C.png' )\n",
    "\n",
    "pyplot.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  使用正确率作评价指标进行调优\n",
    "\n",
    "1.设置候选参数集合 2.生成一个GridSearchCV的实例 3.调用GridSearchCV的fit函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.7747395833333334\n",
      "{'C': 0.1, 'penalty': 'l2'}\n"
     ]
    }
   ],
   "source": [
    "penaltys = ['l1','l2']\n",
    "Cs = [0.001, 0.01, 0.1, 1, 10, 100, 1000]\n",
    "tuned_parameters = dict(penalty = penaltys, C = Cs)\n",
    "\n",
    "lr_penalty= LogisticRegression()\n",
    "\n",
    "#缺省scoring为正确率\n",
    "grid= GridSearchCV(lr_penalty, tuned_parameters,cv=5)\n",
    "grid.fit(X_train,Y_train)\n",
    "\n",
    "print(grid.best_score_)\n",
    "print(grid.best_params_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/listen/anaconda3/lib/python3.7/site-packages/sklearn/utils/deprecation.py:122: FutureWarning: You are accessing a training score ('mean_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "/home/listen/anaconda3/lib/python3.7/site-packages/sklearn/utils/deprecation.py:122: FutureWarning: You are accessing a training score ('std_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xd8VVW2wPHfyk0PLYQEgVBClQ4SCIrggCiIiqCjgmOdgr4nPnUcrDOoOGMZx3GaM6hYsIyI2BhEEERFGYEA0pt0QksICRBCSLnr/XFvYogJuSQ5OSnr+/ncT07Z+9x1KGfl7H3O3qKqGGOMMWcS5HYAxhhjaj5LFsYYY8plycIYY0y5LFkYY4wplyULY4wx5bJkYYwxplyOJgsRGSkiW0Rkm4g8WMr+NiLyhYh8JyJrRWRUsX0P+ettEZERTsZpjDHmzMSp9yxExANsBS4BUoBkYLyqbixW5iXgO1X9l4h0A+aqajv/8jvAAKAlsBDorKoFjgRrjDHmjJy8sxgAbFPVHaqaC8wAripRRoFG/uXGwH7/8lXADFU9pao7gW3+4xljjHFBsIPHbgXsLbaeAiSVKPMY8JmI3AVEAcOL1V1aom6rM31Zs2bNtF27dpUI1xhj6p+VK1ceVtXY8so5mSyklG0l27zGA6+r6nMicj7wpoj0CLAuIjIBmADQpk0bVqxYUcmQjTGmfhGR3YGUc7IZKgVoXWw9nh+amQr9ApgJoKrfAuFAswDroqovqWqiqibGxpabGI0xxlSQk8kiGegkIgkiEgqMA2aXKLMHuBhARLriSxZp/nLjRCRMRBKATsByB2M1xhhzBo41Q6lqvohMBOYDHuBVVd0gIlOAFao6G7gPeFlE7sXXzHSr+h7P2iAiM4GNQD5wpz0JZYwx7nHs0dnqlpiYqNZnYYwxZ0dEVqpqYnnl7A1uY4wx5bJkYYwxplyWLIwxxpTLkoUxxphyWbKoQ26bdxu3zbvN7TCqRF06F2PqAksWxhhjymXJwhhjTLksWRhjjCmXJQtjHFaX+l+SXruGpNeucTuMSqsr5wHVdy5OjjprjAE2HjjmdgjGVJrdWRhjjCmXJQtjjDHlsmRhjDGmXJYsjDHGlMuShTHGmHJZsjDGGFMuSxbGGGPKZcnCGGNMuSxZGGOMKZejyUJERorIFhHZJiIPlrL/eRFZ7f9sFZHMYvv+KCIbRGSTiPxNRMTJWI0xxpTNseE+RMQDvABcAqQAySIyW1U3FpZR1XuLlb8L6OtfvgAYBPTy7/4GuAj40ql4jTHGlM3JO4sBwDZV3aGqucAM4KozlB8PvONfViAcCAXCgBDgkIOxGmOMOQMnBxJsBewttp4CJJVWUETaAgnAIgBV/VZEvgAOAAL8Q1U3lVJvAjABoE2bNlUavDFVpW3edrdDMKbSnEwWpfUxaBllxwGzVLUAQEQ6Al2BeP/+BSIyRFUXn3Yw1ZeAlwASExPLOraphcb9fYNvYaS7cRhjfJxMFilA62Lr8cD+MsqOA+4stj4WWKqqWQAi8ikwEFhcSl1jTDV59G3/DX4tn56jrpwHVN+5ONlnkQx0EpEEEQnFlxBmlywkIl2AaODbYpv3ABeJSLCIhODr3P5RM5Spu7JzC8jOLXA7jCpx68xcbp2Z63YYxlSKY8lCVfOBicB8fBf6maq6QUSmiMjoYkXHAzNUtXgz0ixgO7AOWAOsUdX/OBWrMcaYM3N0pjxVnQvMLbFtcon1x0qpVwDc7mRsxhhjAmfTqpoaySMnyQ6DAm8BniCP2+FUnNdLWEgBHo9Ceu1/KioyPN+3kLEbJMj3CfL8sFza50f77f3a2siShakxCrwFJB9KZu6OuXx6bRA5ocJTb51Hs/BmxEXGFX2aRzU/fT2yOVEhUe4Ffuq47+KZsQsy/T8LP5l7uDgpx1fu7+cFdDhVUC+ggnpB/T8pvlxsH15Qr/jrCWjx9VKOU0bd4scv2leiTgd/Y/Hen15UiT8w+eGnyA/rUnI7pewvq0xZxyt9ufOpU77zuKrUp/lrlc6nTuENdf5hUEsWxlWqyob0DXyy4xPm7ZrH4ZOHiQqJovseJT5daf6rCaRmp5Kancqe43tYcXA5J3KOE1wAQV4I9vp+NgqKJDYshrjQpsSGNiUmpAkxIU1oGtKYpsGNiQ5uSENPFEEFXrSgAM3Lh4J8ND8fzS9A8/PAv10LSqzn5UJ2JnoiA83OQLOPQvYxNOc4mpMFebn+C6z/Yk0w6glHg8JQ6cDJtMOoVwhv1tx3zAJv0af4Ov7YynzA3ElBgniCEI8HPEH+5SDweIqWJSiIk0d878ZGxLUo/Bv84WdR3Frsh5bYVvzkSmwrLF9qveLbih/nDN+Pt5SYfD/1lC9x5B05UcofRu2ipwTxWLIwddTOozuZu3Muc3fMZc/xPYQEhTA8ZhBXSgc67DzFoY9fJyQfQpLfRPPzId93EcfrLeOIx/2fXaXuPeb/VIgoIiBB6vvlNQjfhTQ4GAlugISGIiFhEB6BhEYgoeEQ7EGCQwjyeDh5aAkKNO5+HuLxICHB4AlGgoORYI/vOP51Xz3/ekiw72IdHIL4t5daLyTYd5EP9u8rvly0XvwYHiQkxBdL4XpQYM+6zB3eDYBR87+s6J9mjVB0HgvXuxxJ5RWeSyeHv8eShak2h04cYt6ueXyy4xM2HdlEwxzhyqyODEkbSPOth8nbvAi8CzkaEgICJyKg7ZgxpV5US14sS15UNSiILM0hM/8YmflZZOQeIT37AEdyUknPPcLh/GOkaQ7HgrwUBOH7eKBAIFy8xAAxwWE0C2tE88g44hq2Ii66A82bnUtcsx40jYorty/lZG4Bq/Zk8PaUJzgVFEbXUb8gPCSI8BAPYSEewoN9y76PfznYU6zMD9tCPIKNpWncZMnCOOroqaMs2L2AuTvnsmVHMufu9TIqLYYH9jYlcnca6CYkNJSQ3r1pfMcdRA7oT0Tv3sy7sh8ASY88HNgXqUJW6ml9Bo0zdtGqsC/h2D5Oa8bwhEF0W7Ibtya1cXNSI5twKCySVE8QqeSTeiqT1OxUtmcf4vCJ7ynI2gwHPi+qHizBxETE0Dzyh/6TJmHNyDoRxYH0UL7f72FzipCXH4K0GE6oN48vvtyGt4KtBUHCD4kluFjCCQk6LcEUJp6w4JJJqMT+Eokp/LRjeQgLDiIoyJKT+YElC1PlTuaf5Ku9X7FozQccW7aUc3fnc9O+EM455HuSRsKPE9G3D5Gjryeqf3/Ce/UiKCys/APnnoDMPad3IBfvWM7LPr18wxbQpC20uxCi2/k/bX0/G5wDQUFEAu38n7IUeAtIz0knNTuVQ9mHivpQUo4fZMeRfSzP2MTx/G9QyfmhUgSEd4JoTwMapGcRnq80Prenr19DwauKV319NqcvF+7TonLFyxeu56iSXbJ+juLN8Zfx+rcF+HdWGhHfi1gSJASJECTgHdsCASa/emUljuy+grG+Ppfafh7gO5cGWRGMcvh7LFkAt83zvSf/2sjXXI6k9srz5rF87adsWDiT/FVr6bwrj9uO+HdGhBPVL5HIG/oT2b8/ET26I6GhZzxeZHg+reKy4f1f/ZAYTqSeXigkynfhb9oeOgw7PRk0aQMhEVVybp4gD3GRcYTShP2HmrFtZwuW7Uxn4/5jeBVCPUH0bt2Y89pG0rFlAXHRORzN8yeXE4dYt/Md8jzQOKxxlcRzVvwJpsD7QxLyeqGgaNn3s0ApWi5tf+G2k/n5gBAW1KD6z6UK5ebnARBay88DfOcSluf8XaAlizqkugffO5WSwuZF77Pv6wVErd9JXIaXwUBuRAhBvXsTe+FwogYMILxbN19HaqDyc0nslk5URD7sWepLAJ1HFLs78H8iYxx9Zj896xTLdx5h2c4jLN2RzpZDx1GFsOAg+rZpwl3DOpHUvinntYkmPKTs/ou5v30bgFEPT3Us1uqS9No1ACy59Z1yStZsdeU8wHcuxxs5/z2WLExAVJW8vXvJTk7m4Defk52cTMThLEKB2HBI79KcY9ddSLfh19KgWw/fEzgVtfSfNIjMJ3lDDP3fW1dl51Ce1OM5LNtxhGU701m24wjfp2YBEBHioV/baC7v2YKk9jH0bt2YsODAz+/xn3UFcLyZwBgnWbIwpVJVcnfuIjs5mezkZI4vX4qmHgbgaCRsbh3EyZ90pP1PrmTQ4PE0CGtYNV987AAsfpZD6eGkZYRXzTHLcODoydOSw47Dvmfuo0I9JLZrytjzWpGUEEPPVo0JDbbp6k39ZsnCAP7ksG0bJ/zJIXvFCgrSfMkhq2EIa+Lz2dg3CO3TjaSBVzM6YQRNw5tWfSALH4WCPDbtbFLlh957JJtlO4+wbEc6y3YeYc8RX4d4w/BgBrRryrgBrUlKiKF7y0YEeyw5GFOcJYt6Sr1eTm3dSvbyYskhIwOAvGaN2Z4QxteJwaxvrTTo0IFR7S/nvoTLaNWglXNB7VkKa9+Fwb8h+6vKtSWrKrvTs4vuGpbtPMK+zJMANIkMYUC7ptxyQTuSEprStUUjPPaYqDFnZMmintCCAnI2bS5qVspeuRLv0aMABLdqyfHEzqxocZKPG25jb8MsWjZozKj2v+SOhMvoHN3Z+QC9BTD3N9AoHgb/mte3vw8E3s6vqmxPO1EsOaRz6Jhv/J+YqFCS2jdlwpD2JLVvSue4hvYOgTFnyZJFHaV5eeRs3Eh2cjInkpM5uXIV3ixfh21I2zY0GH4xhzo3Y2HT/XyUtYTjuSuJDotmRLurubz95fSO7V29bwyvfB0OroOfvgah5Q8K6PUq36dmnXbncDjLlxziGoaR1D6GpISmDGzflA6xDeztZ2MqyZJFHeLJ9xJ1PI89v/gl2d99h2b72uRD27en0eWXE9k/kX0dG/PR8WV8uutTUrNTiTwWycVtLmZU+1EktUgiJCik+gPPPgKLnoB2g6H72FKLeL3KpoPHiu4alu88Qka271n5lo3DGdypGUkJTUlqH0O7mEhLDsZUMUsWdcCpbdtIn/YKLXefQID81FSajBlD5ID+RCYmsi/0BB/tnMvcHS+ya9kugoOCGdxqMJP6T+Ki+IuICK6al9cqbNHvIecYXPZM0XsTqsKJ3Ja8vHhHUXI4luN7A7x10wgu7trcf+cQQ3x0hCUHYxxmyaIWy171HenTppG1aBESEUFW4xCONQ5l+H9mk5adxse75jF36V2sT1+PIPQ/pz+3dr+V4W2Hu/M2cWkOrIWVr0H/X0Hz7kWb1x2+jZzjHdi4bxMJzaIY1bMFSe2bkpQQQ8smLic3Y+ohR5OFiIwE/gp4gGmq+nSJ/c8DQ/2rkUCcqjbx72sDTANa4xviZpSq7nIy3tpAVcn66ivSX57GyZUr8TRuTLM77yT6xp/xya2XsrZ1ATM++yXLDyxHUbrFdOM3ib9hZLuRNI9q7nb4p1OFT++HiGgY+lDR5k/WHiDneAcio9fxxf/cQ/NGzr5vYYwpn2PJQkQ8wAvAJUAKkCwis1V1Y2EZVb23WPm7gL7FDvEG8AdVXSAiDYCyJjKoFzQvj2Nz55I+7RVOff89wS1b0Pzhh2l8zdWsztrMXzb8iU9H55AXDG2yDnBH7zu4LOEyEhonuB162da9B3u+hSv/5ksY+IbYmPzxeoLD02gQu8IShTE1hJN3FgOAbaq6A0BEZgBXARvLKD8eeNRfthsQrKoLAFQ1y8E4azRvdjaZs94n/fXXyN9/gLBOnWj5zNPkDk3iw92f8NFn49h1bBdRIVH02+2h/85gfvXGnJrfhn/qOHz2O2jZF/reVLT5sf9s5FhOHo1bf42IG1PGGWNK42SyaAXsLbaeApQ64a2ItAUSgEX+TZ2BTBH5wL99IfCgqhY4F27Nkp+RQcZbb5Px9tsUZGYSkdiPuN/9llXt4fntH/L1R49SoAWcF3cev+j5Cy5teylfj/kJQM1PFACL/wRZB2Hc2+CfpW3e+oP8Z81+7rukM2/tyXQ5QGNMcU4mi9KuWGX9qjgOmFUsGQQDg/E1S+0B3gVuBV457QtEJgATANq0aVP5iGuAvH37SH99OpmzZqEnT9Jg2DByx1/Oh5FbmL39CQ7vOUxMeAw3d7+ZsR3H1uxmprIc3gbfvgB9fgbxiQBknMjltx+tp3vLRtzxkw689YbLMRpjTuNkskjB1zldKB7YX0bZccCdJep+V6wJ6yNgICWShaq+BLwEkJiYWKvbLHK2bCX9lWkc+2QuiBA1aiQbRnZixqklrNrxAB7xMDh+MGM7jmVw/GB33oeoCqow70HfXBPDHyvaPGXORjKzc3nj5wMIsXGZjKlxnEwWyUAnEUkA9uFLCDeULCQiXYBo4NsSdaNFJFZV04BhwAoHY3WFqnJy5UrSX55G1ldfIZEReK8ZyScDgvjg2GJO7J1H20Ztuee8exjdYTSxkbFuh1x5W+fBtgVw6R+gQRwACzce4sPv9nH3xZ3o1rIaBuY3xpw1x5KFquaLyERgPr5HZ19V1Q0iMgVYoaqz/UXHAzNUVYvVLRCR3wCfi68BfiXwslOxVjf1esn64gvf46+rVyPRTdg3bgivdjnAutz5hGeGc2m7SxnbcSz9mvcLuA9ixl2+9xRGOBl8ZeTl+O4qmnWBpNsBOJqdx8MfruPccxpy59COLgdojCmLo+9ZqOpcYG6JbZNLrD9WRt0FQC/HgnOB5uZydM4npL/yCrnbt1NwTjOWjuvGtDY7OOH5Lz0a9uB3nX7HZQmX0TC0iuaHqEm+/btvetSbPgKPrxntiU82kn4il1dv7W9zRhhTg9kb3NWgIOsEme+9x5Hp08k/eJDjbWL44NqmfNo+g4YRytj24xjbaWz1jO7qlqMp8PWfoeuV0MH3HuYXW1KZtTKFiUM70qPV6W+Ut8v9jRtRGmPKYMnCQflHjnDkzTfJePvfeI8dY2/Hxrx5nYc17Y9yfqsLeKbTWIa1HkaoJ9TtUJ332W9Bvb6+CuBYTh4Pvb+Ozs0bcNfF1vxkTE1nyQIY9/cNvoWRVXO83JQUjrz6GhmzZqF5uXzXJZRZYzxkd27MmI638EzHq2jZoGXVfFltsPNr2PAh/OQhiG4LwJOfbCL1eA4v3jTorOazNsa4w5JFFcrZvJmDL/6L7PkL8Ap82QM+PT+Mbn0vYVKnsQxsMZAgqWft8gX58OkD0KQNDLobgMVb05iRvJc7LupA79ZVP31qTdOthT3hZWo/SxaVpKqcWL6cnS/8meDlazkZCgv6C5uGd+SSxOt5O+FymoTX/QtimVa8Aqkb4Pq3ICSCrFP5PPTBOjrERnHP8E5uR2eMCZAliwpSr5eUuR9wYOoLNNx2kKxI+HxoOMHXXMGVfcZxf0y32jHshpNOHIYv/gDth8K5VwDw1NxN7D96kll3XEB4iDU/GVNbWLI4S7k5J1g5/c/ovz8i+lA2J5rAt9cm0GH8L7mv82XuTyRUk3z+OOSeKJrU6L/bDvP2sj38anAC/dpGux2dMeYsWLII0PZ961kz7VlazFlBk+Ne9rQIZs/Eixl4w68Z2rS92+HVPPtWwao34fw7IbYLJ07lc//7a0loFsV9l3ZxO7pq9drI19wOocosu+19t0OoEnXlPKD6zsWSxRlk52Wz8LtZpE5/jT7fHKTrKUjp0hS97UaGXflLQjy1dHwmp3m9MHcSRMXCRQ8A8Md5m9mXeZKZt59vzU/G1EKWLEpQVdakrWHBf98k8r0FXLg6jw5eyEjqTKuJk+iaeKHbIdZ8a96BfStgzL8gvBFLd6Qz/dvd3DaoHf3bNXU7OmNMBViy8MsKU6ZvmM6yL9+h38I9jNqsqCcIvWwoHf93EmHta+FQ4G7IOQoLH4X4/tBrHCdzC3jg/bW0aRrJpBH1q/nJmLqk3ieL1OxUpg/MQfILuPK+Z7hrl1IQGUbj266n+a2/ICQuzu0Qa5ev/uh7Cupn70FQEM/O38ju9Gze+dVAIkMD/+f27u3nOxikMeZs1ftkEXE4i5s/y6dtmkJMNLH33Ub0uHF4GtbBgfyclrYFlk2F826Gln1ZsesIr/13JzcNbMv5HWLcjs4YUwn1Plk0OCee5seDSI8N5oKFXxAUFuZ2SLWTKnx6P4RGwcWTyckr4P5Za2nVJIIHLzvX7eiMMZVU75OFhIaS2ioSwBJFZWz6D+z4Ei57FqKa8ee5m9hx+ARv/zKJqLB6/8/MmFqvng1UZByRmw3zH4G47pD4c1btyWDa1zsYP6ANgzo2czs6Y0wVsF/5TOUt+Ssc3QO3fkKOV5j03hrOaRTOw6Os+cmYusLuLEzlZOyGJX+B7ldDuwv56+ffsz3tBE9d04uG4fbSojF1hSULUznzHwYJgkt/z5q9mbz41XauS4znos6xbkdmjKlCjiYLERkpIltEZJuIPFjK/udFZLX/s1VEMkvsbyQi+0TkH07GaSpo+yLYPAcG38epqHOYNGsNcQ3DeeTybm5HZoypYo71WYiIB3gBuARIAZJFZLaqbiwso6r3Fit/F9C3xGGeAL5yKkZTCQV58OmDEJ0AF9zFPxZtY+uhLF67tT+NI6z5yZi6xsk7iwHANlXdoaq5wAzgqjOUHw+8U7giIv2A5sBnDsZoKmrZi3B4C4x8mvWHcvjnl9u5+rxWDD3X3ng3pi5yMlm0AvYWW0/xb/sREWkLJACL/OtBwHPApDN9gYhMEJEVIrIiLS2tSoI2ATh+CL58GjpdSm6HS5k0ay0xUaE8ekV3tyMzxjjEyWRR2jRxWkbZccAsVS3wr/8vMFdV95ZR3ncw1ZdUNVFVE2NjrUO12ix8DPJzYMRT/PPLbWw6cIw/jO1J40hrfjKmrnLyPYsUoHWx9XhgfxllxwF3Fls/HxgsIv8LNABCRSRLVX/USV4VZtzl+414hBMHr2v2Loc1/4ZB97ApL45/LPqGq/q05JJuzd2OzBjjICeTRTLQSUQSgH34EsINJQuJSBcgGvi2cJuq/qzY/luBRKcShTkL3gLfpEYNW5B34X1MmraGJpEhPHalNT8ZU9c51gylqvnARGA+sAmYqaobRGSKiIwuVnQ8MENVy2qiMjXFd2/CgdVwyRO8+O0h1u87xu/H9CA6KtTtyIwxDnN0uA9VnQvMLbFtcon1x8o5xuvA61UcmjlbJzPg8ynQ5gK2xo3gb+8u4fJeLRjZo4XbkRljqoG9wW0C88VTcDKD/BFPM2nWWhqEBzNltDU/GVNfWLIw5Tu0AZKnQeLPefn7BqxJOcqUq7oT08CGdDemvggoWYhID6cDMTWUKsy9H8Ibs6PnPTy/cCsju5/D5T2t+cmY+iTQO4upIrJcRP5XRJo4GpGpWTZ8ALu/wTvsd9w3Zw+RoR6eGNMDkdJeozHG1FUBJQtVvRD4Gb73JlaIyL9F5BJHIzPuyz0Bn/0OzunFq9mD+W5PJo+P7k5sQ2t+Mqa+CbjPQlW/B34LPABcBPxNRDaLyNVOBWdc9vVzcGwf+y6YwrMLtjG8a3NG927pdlTGGBcE2mfRS0Sex/e+xDDgSlXt6l9+3sH4jFvSt8N//472vI57/htGWHAQT4615idj6qtA7yz+AawCeqvqnaq6CkBV9+O72zB1zfyHwRPKu9G/InlXBo9e2Z24RuFuR2WMcUm5L+X556XYq6pvlra/rO2m+r028rWqOdDWz2DrPDIG/ZbHvkhnaJdYrj6v1AGDjTH1RLl3Fv6RYGNExMZ0qA/yT8G8B9GYTty5PYmQoCCevLqnNT8ZU88FOtzHbmCJiMwGThRuVNU/OxKVqZDrX/SNxfju7edX/CBL/wlHtvN5v3/y3yXH+eM1vWjROKKKIjTG1FaBJov9/k8Q0NC5cIyrju2Hr54lu/0I/i85hiGdm3JtYrzbURljaoCAkoWqPu50IKYGWDAZ9ebzcPZ4gkR4ypqfjDF+ASULEYkF7ge6A0WPxKjqMIfiMtVt939h3Xts6DCBjzaE8oex59KqiTU/GWN8An109m1gM755sh8HduGb3MjUBd4CmHs/+Q1bceu2wVzQIYYbBrRxOypjTA0SaLKIUdVXgDxV/UpVfw4MdDAuU51WvAqH1vGv0NvI1lCeuaaXNT8ZY04TaAd3nv/nARG5HF9nd53p+ayy9xNqoxPpsOj3HIoZwHP7ujLlqnNp3TTS7aiMMTVMoMni9yLSGLgP+DvQCLjXsahM9Vn0BHrqOLefvJ6khBhuTGrrdkTGmBoo0Keh5vgXjwJDAz24iIwE/gp4gGmq+nSJ/c8XO14kEKeqTUSkD/AvfEmpAPiDqr4b6PeaAB1Yg658nQUNx7AlM555P+1FUJA1PxljfizQp6FeA7Tkdn/fRVl1PMALwCVACpAsIrNVdWOx+vcWK38X0Ne/mg3crKrfi0hLYKWIzFfVzEDiNQHwT2p0KjSa36SNYtIVXWgbE+V2VMaYGirQZqg5xZbDgbH4+i3OZACwTVV3AIjIDOAqYGMZ5ccDjwKo6tbCjaq6X0RSgVjAkkVVWTsT9i7lSe6gS7t4br2gndsRGWNqsECbod4vvi4i7wALy6nWCthbbD0FSCqtoIi0xfdY7qJS9g0AQoHtgcRqAnDqOLpgMrtCuzAzewif/rS3NT8ZY84o4MmPSugElPcgfmlXnx81ZfmNA2b5By384QAiLYA3gdtU1fujLxCZICIrRGRFWlpaAGEbAL76I5J1kHuP/4z7Lu1KQjNrfjLGnFmgkx8dF5FjhR/gP/hmzDuTFHzTsBaKp+ymq3HAOyW+sxHwCfBbVV1aWiVVfUlVE1U1MTY2NpBTMYe/R5f+i48YirRO5OcXJrgdkTGmFgi0GaoigwcmA51EJAHYhy8h3FCykIh0AaKBb4ttCwU+BN5Q1fcq8N2mNKropw+QQyjP5F/Pmz/tjcean4wxAQj0zmKs/z2LwvUmIjLmTHVUNR+YCMzHNx3rTFXdICJTRGR0saLjgRmqWryJ6jpgCHCriKz2f/oEeE6mLFvmIts/59lTV3Pz8AF0jGvgdkTGmFoi0KehHlXVDwtXVDVTRB4FPjpTJVWdC8wtsW1yifXHSqn3FvBWgLGZQOSdpODTB9lFPKvP+SkzB1vzkzEmcIF2cJdWLtBEY2qC//4Dz9E9PJ5/K09d249gT0WfbTAmxqlqAAAaoElEQVTG1EeBXjFWiMifRaSDiLT3v3m90snATBXK3EvB4j/xScEABgwbQ5dzbP4qY8zZCTRZ3AXkAu8CM4GTwJ1OBWWqVu6nD5NX4OW9pndw+0Ud3A7HGFMLBfo01AngQYdjMU7Y8RWhW2bzfP61PDDuEkKs+ckYUwGBPg21QESaFFuPFpH5zoVlqkRBHlkf3cdebyzBg/+Pri0auR2RMaaWCrSTulnxQfxUNUNE4hyKyVTQ5PRJ/qVvAMheMpUGx77nuQYP89DFPdwLzBhT6wXaJuEVkaLhPUSkHWUP3WFqgqw05MunWeztxTU33E5osDU/GWMqLtA7i0eAb0TkK//6EGCCMyGZqrD//QeJLTjJ9vN+y5D4JuVXMMaYMwi0g3ueiCTiSxCrgY/xPRFlaqCs7ctouXMW74WN5YYrhrsdjjGmDgh08qNfAnfjGwxwNTAQ31hOw5wLzVSIKkdm3U22NqHruCcIC/a4HZExpg4ItCH7bqA/sFtVh+Kb0c7GBK+BMvODaXNyE8s73k2P9q3Lr2CMMQEINFnkqGoOgIiEqepmoItzYZmKOOX1cK53O+s95zL8+v9zOxxjTB0SaAd3iv89i4+ABSKSQfnTqppqdiLXSxPJIvXyPxEeakN3GWOqTqAd3GP9i4+JyBdAY2CeY1GZs7b+648ZHLSWL/Q8hp432O1wjDF1zFn/+qmqX5VfylSnE8cyiFl0H3s0jiYhBeVXMMaYs2RvatUB69/4Nc29h9kWlEBo0I+mKjfGmEqzZFHLbVjyCUmHP2B58+toHpLtdjjGmDrKkkUtlp11lCYL7yVFzqH3Lc+5HY4xpg5zNFmIyEgR2SIi20TkR0Oci8jzxebY3ioimcX23SIi3/s/tzgZZ2217o37aKWHOHrpX4iIsgmNjDHOcez5ShHxAC8AlwApQLKIzFbVjYVlVPXeYuXvwveyHyLSFHgUSMQ3YOFKf90Mp+KtbTYtm0//Q7NYFnsNSedf5nY4xpg6zsmH8QcA21R1B4CIzACuAjaWUX48vgQBMAJYoKpH/HUXACOBdxyMt9Y4eeI4Debdw8GgOHrc8uei7VNingV80xkaY0xVcrIZqhWwt9h6in/bj4hIWyABWHS2deujNW9MorXu58jFfyKqoY0oa4xxnpPJQkrZVtYcGOOAWapa+JJAQHVFZIKIrBCRFWlp9WOoqs3JCxlwcAbLYsbQ48LRbodjjKknnEwWKUDxkeziKXuIkHGc3sQUUF1VfUlVE1U1MTY2tpLh1nw5J08Q8endpEozut/yF7fDMcbUI04mi2Sgk4gkiEgovoQwu2QhEekCROMb8rzQfOBS/1zf0cCl/m312ndvPEhbbwppQ/9Ig0bRbodjjKlHHOvgVtV8EZmI7yLvAV5V1Q0iMgVYoaqFiWM8MENVtVjdIyLyBL6EAzClsLO7vtq66ksG7H+T5U2vYMBFV7sdjjGmnnF0aFJVnQvMLbFtcon1x8qo+yrwqmPB1SKncrIJnTORw9KUc2/5m9vhGGPqIXuDuxZY9ebDtPPu5eBFz9CoSYzb4Rhj6iFLFjXcttVf0z9lOssbX0bvode6HY4xpp6yZFGD5Z7KwTN7IhnSmC63/N3tcIwx9Zglixps5Vu/JcG7i30XPkXjpnX/0WBjTM1lyaKG2r5uKYl7XiW50SX0GT7e7XCMMfWcJYsaKC/3FHz0PxyVhnS6+QW3wzHGGEsWNdGKtyfToWAHe87/PU2aNXc7HGOMcfY9C3P2dm5Mpt+ul1nZaCj9RtzkdjjG1Ap5eXmkpKSQk5Pjdig1Vnh4OPHx8YSEhFSoviWLGiQ/L5f8D+4gS6JIuMman4wJVEpKCg0bNqRdu3aIlDYOaf2mqqSnp5OSkkJCQkKFjmHNUDVI8r8fp1P+NnYOmELTOBuR3ZhA5eTkEBMTY4miDCJCTExMpe68LFnUELs3r6TfjqmsihpCv1G3uR2OMbXO2SaK61/8lutf/Lb8gnVEZROpJYsaoCA/n5xZ/8MJiaDNTf90OxxjTAU0aNCgaHnkyJE0adKEK664otSyd955J3369KFbt25ERETQp08f+vTpw6xZs87qO1etWsW8efMqFXegrM+iBkie8QQD87ewIvFZEs9pXX4FY0yNNmnSJLKzs3nxxRdL3f/CC74+yV27dnHFFVewevXqCn3PqlWrWL9+PSNHjqxwrIGyOwuX7f1+DX2+f4HvIi+g36hfuh2OMaYKXHzxxTRs2LBCdb///ntGjBhBv379GDJkCFu3bgVgxowZ9OjRg969ezN06FBOnjzJlClTePvttyt0V3K27M7CRQX5+WTNvINGEkrrG6ciQZa7jamsx/+zgY37j5VbbuMBX5lA+i26tWzEo1d2r3RsgZgwYQLTpk2jQ4cOLFmyhIkTJ/LZZ5/x+OOP8+WXX9K8eXMyMzOJiIhg8uTJrF+/nr/8xfmZMy1ZuCh55lMMzNtIct+n6N+ybaWP9+7t51dBVMYYt2RmZrJ06VKuueaaom35+fkADBo0iJtvvplrr72Wq6+u/gnQLFm4JGXbenpv+RtrIpNIvPIOt8Mxps4I9A6g8I6iJv2Spao0a9as1D6Ml19+mWXLljFnzhx69+7N2rVrqzU2a/dwgbeggGMz7yBPgmlx44vW/GSMASA6OpoWLVrw4YcfAuD1elmzZg0AO3bsYODAgTzxxBNER0ezb98+GjZsyPHjx6slNrtKuSD5vT/SLXcdm3s/RFyrir1NaYypuQYPHsy1117L559/Tnx8PPPnzw+47owZM5g6dSq9e/eme/fuzJkzB4B7772Xnj170rNnT4YPH06PHj0YNmwYa9asoW/fvrW7g1tERgJ/BTzANFV9upQy1wGPAQqsUdUb/Nv/CFyOL6EtAO5WVXUy3uqwf+dmem56nrURifS/aqLb4RhjqkhWVlbR8tdffx1QnXbt2rF+/frTtrVv377U5DJ79uwfbYuNjWXFihVnGWnFOJYsRMQDvABcAqQAySIyW1U3FivTCXgIGKSqGSIS599+ATAI6OUv+g1wEfClU/FWB/UWkPHOBBoRRNzPrPnJGDfVpL6K2sDJq9UAYJuq7lDVXGAGcFWJMr8CXlDVDABVTfVvVyAcCAXCgBDgkIOxVovk9/9M99w1bOh5P+e07uh2OMYYEzAnk0UrYG+x9RT/tuI6A51FZImILPU3W6Gq3wJfAAf8n/mquqnkF4jIBBFZISIr0tLSHDmJqnJg9xa6r/8T68L6MuDqe9wOxxhjzoqTyaK0UatK9jkEA52AnwDjgWki0kREOgJdgXh8CWaYiAz50cFUX1LVRFVNjI2tuXNUq9fL4X/fgaDEjLfmJ2NM7ePkVSsFKD7QUTywv5QyH6tqnqruBLbgSx5jgaWqmqWqWcCnwEAHY3VU8od/o+epVazr9htatuvidjjGGHPWnEwWyUAnEUkQkVBgHFCyO/8jYCiAiDTD1yy1A9gDXCQiwSISgq9z+0fNULXBoZTtdF37NBtCe9H/p/e5HY4xptBrl/s+JiCOJQtVzQcmAvPxXehnquoGEZkiIqP9xeYD6SKyEV8fxSRVTQdmAduBdcAafI/U/sepWJ2iXi8H37odD16ajHuJII/H7ZCMMQ6p7iHKP/zwQ5599tlKxx0oR9+zUNW5wNwS2yYXW1bg1/5P8TIFwO1OxlYdVsz+J/1zkll67v0MbN/V7XCMMdWkqoYoz8/PJzi49Mv02LFjqybYAFlPq0PS9u+iy+on2RjSgwHXPeh2OMaYalSZIcovvPBCHnnkEYYMGcI//vEPPv74Y5KSkujbty+XXnopqam+NwymTZvGPff4nqy88cYbufvuu7ngggto37590XAhVckGEnSAer3se+sOztVcGl4/1ZqfjKlOnz4IB9eVX+6gfyC+QPotzukJl/1oAArHHDt2jMWLFwOQkZHB6NGjERGmTp3Kc889xzPPPPOjOqmpqSxZsoR169Zx3XXXVfmdhyULB6yc8xKJ2d+ytPOvGdixp9vhGGNqmXHjxhUt79mzh+uuu46DBw9y6tQpOnfuXGqdMWPGICL06tWLffv2VXlMliyq2OGDe+i0agqbg7vS//pH3A7HmPon0DuAwjuK2z5xLpYKioqKKlq+8847efjhhxk1ahQLFy7k6adLP7+wsLCiZSeG0bM+iyqkXi973/wfwjWXiGun4imjY8oYYwJ19OhRWrVqhaoyffp01+KwZFGFVn36Kn1PfMN3Hf6Htl36uB2OMcYllRmivKTHHnuMsWPHctFFF9G8efMqjPLsSB0Y9RuAxMREra6hekuTfiiFoH+dT1rwObR/YAnBIaGuxWJMfbNp0ya6dj3Lx9NrcDOUU0r7cxKRlaqaWF5dayepIrvf/F96aDZHr5lqicKY2qAeJYmqYM1QVWDVvNc5L+srViXcTruu/dwOxxhjqpwli0rKSDtA26WT2ebpQL8bHnU7HGOMcYQli0ra/sadNNQsgsb+i5DQsPIrGGNMLWTJohK+++wtEo9/zsq2v6R9jyS3wzHGGMdYsqigo+mHaP3fR9juSSDxxifcDscYc5Zum3cbt827ze0wag1LFhW09Y2JNNbjcNU/rfnJGFM0RPnq1as5//zz6d69O7169eLdd9/9UdmqGKIcYNWqVcybN69K4i+PPTpbAWsWzaD/0c9Y2voXDOx1gdvhGGNqkMjISN544w06derE/v376devHyNGjKBJkyZFZQIdorw8q1atYv369YwcObJKYj8Tu7M4S0czDtNi8UPsDGrLeTc96XY4xpgapnPnznTq1AmAli1bEhcXR1paWsD1v//+e0aMGEG/fv0YMmQIW7duBWDGjBn06NGD3r17M3ToUE6ePMmUKVN4++23K3RXcrbszuIsbZl+F+dpJkevnE5oWLjb4RhjSnhm+TNsPrK53HKFZQLptzi36bk8MOCBs45l+fLl5Obm0qFDh4DrTJgwgWnTptGhQweWLFnCxIkT+eyzz3j88cf58ssvad68OZmZmURERDB58mTWr1/PX/7yl7OO7WxZsjgLa7+YxYDMuXzb6hbO7zvE7XCMMTXYgQMHuOmmm5g+fTpBQYE14mRmZrJ06VKuueaaom35+fkADBo0iJtvvplrr72Wq6++2pGYz8TRZCEiI4G/Ah5gmqr+aGxdEbkOeAxQfHNt3+Df3gaYBrT27xulqrucjPdMjh89QvOv7md3UGv63vSUW2EYY8oR6B1A4R3FayNfq/IYjh07xuWXX87vf/97Bg4cGHA9VaVZs2al9mG8/PLLLFu2jDlz5tC7d2/Wrl1blSGXy7E+CxHxAC8AlwHdgPEi0q1EmU7AQ8AgVe0O3FNs9xvAs6raFRgApDoVayA2Tr+bZnqEnFF/IzwiqvwKxph6KTc3l7FjxxbdBZyN6OhoWrRoUTQtqtfrZc2aNQDs2LGDgQMH8sQTTxAdHc2+ffto2LAhx48fr/JzKI2THdwDgG2qukNVc4EZwFUlyvwKeEFVMwBUNRXAn1SCVXWBf3uWqmY7GOsZrVv8MUlHZpPc4ga6JA5zKwxjTC0wc+ZMFi9ezOuvv170SOzZPO00Y8YMpk6dSu/evenevTtz5swB4N5776Vnz5707NmT4cOH06NHD4YNG8aaNWvo27dvre7gbgXsLbaeApR8zbkzgIgswddU9ZiqzvNvzxSRD4AEYCHwoKoWOBhvqbKOZdBs0X3slZb0ufmP1f31xphaIisrC4Abb7yRG2+8MaA67dq1Y/369adta9++fanzX8yePftH22JjY6muqRmcTBZSyraSk2cEA52AnwDxwNci0sO/fTDQF9gDvAvcCrxy2heITAAmALRp06bqIi9mwxu/pr8eZuuo92gd2cCR7zDGVD8n+irqMieboVLwdU4Xigf2l1LmY1XNU9WdwBZ8ySMF+M7fhJUPfAScV/ILVPUlVU1U1cTY2NgqP4H1S/5D0uEPWH7O9Zw74JIqP74xxtQWTiaLZKCTiCSISCgwDih5H/URMBRARJrha37a4a8bLSKFGWAYsNHBWH8kO+so0QvvI0Va0PvmP1XnVxtjTI3jWLLw3xFMBOYDm4CZqrpBRKaIyGh/sflAuohsBL4AJqlqur9v4jfA5yKyDl+T1stOxVqatdPvo5Ue4tilzxMR1bA6v9oYY2ocR9+zUNW5wNwS2yYXW1bg1/5PyboLgF5OxleWjUvnMSB1FsvifkrS+Ze5EYIxxtQoNjZUCSdPHKfR/Hs4GBRHz1v+7HY4xhiH7L7pZnbfdLPbYdQalixKWPPGJOL1AEeGP0dkg8Zuh2OMqSWqe4jyDz/8kGeffbbK4i+PjQ1VzOblCxhwcAbLmo0hadCVbodjjKmFqnKI8vz8fIKDS79Mjx07tuqDPwNLFn452VlEzrubVGlG91ucH8HRGFM3de7cuWi5+BDlxZPFmVx44YVcdNFFfP3111x99dUkJCTw5JNPkpubS2xsLG+99RZxcXFMmzataMTZG2+8kZiYGJKTkzl48CDPPfdclScTSxbA9S9+yxWpU7nJu491w17nnEbRbodkjKmgg08+yalN5Q9RnrPZVyaQfouwrudyzsMPn3UsFRmiHHwDES5evBiAjIwMRo8ejYgwdepUnnvuOZ555pkf1UlNTWXJkiWsW7eO6667zpKFE1qe2MAN+R+zvOkVDBhSvbd2xpi6qSJDlBcaN25c0fKePXu47rrrOHjwIKdOnTrtzqW4MWPGICL06tWLffv2VSr20tT7ZHEqJ5v/y3yWNGnCubf8ze1wjDGVFOgdQOEdRds336jyGCo6RHmhqKgfRra+8847efjhhxk1ahQLFy7k6ad/NNMDAGFhYUXLvrcSqla9fxoq/aBvrMN1QV1o1CTG5WiMMbVdZYYoL83Ro0dp1aoVqsr06dOrIMKKqfd3Fi3bdWFci2loUDA2+pMxprIKhyhPT0/n9ddfBygarrwiHnvsMcaOHUt8fDwDBgzgwIEDVRht4MSJ2xU3JCYmakWH6r3+xW8BePf286syJGNMNdm0aRNdu3Y9qzpONkPVVKX9OYnISlVNLK9uvb+zAEsSxtRH9SlJVIV632dhjDGmfJYsjDHGlMuShTGmTqgr/a9OqeyfjyULY0ytFx4eTnp6uiWMMqgq6enphIeHV/gY1sFtjKn14uPjSUlJIS0tze1Qaqzw8HDi4+MrXN+ShTGm1gsJCSEhIcHtMOo0a4YyxhhTLksWxhhjymXJwhhjTLnqzHAfIpIG7K7EIZoBh6soHDfVlfMAO5eaqq6cS105D6jcubRV1djyCtWZZFFZIrIikPFRarq6ch5g51JT1ZVzqSvnAdVzLtYMZYwxplyWLIwxxpTLksUPXnI7gCpSV84D7FxqqrpyLnXlPKAazsX6LIwxxpTL7iyMMcaUy5KFn4g8ISJrRWS1iHwmIi3djqmiRORZEdnsP58PRaSJ2zFVlIhcKyIbRMQrIrXuyRURGSkiW0Rkm4g86HY8lSEir4pIqoisdzuWyhCR1iLyhYhs8v/butvtmCpKRMJFZLmIrPGfy+OOfZc1Q/mISCNVPeZf/j+gm6re4XJYFSIilwKLVDVfRJ4BUNUHXA6rQkSkK+AFXgR+o6oVmzvXBSLiAbYClwApQDIwXlU3uhpYBYnIECALeENVe7gdT0WJSAughaquEpGGwEpgTG38exERAaJUNUtEQoBvgLtVdWlVf5fdWfgVJgq/KKDWZlFV/UxV8/2rS4GKDzXpMlXdpKpb3I6jggYA21R1h6rmAjOAq1yOqcJUdTFwxO04KktVD6jqKv/ycWAT0MrdqCpGfbL8qyH+jyPXLksWxYjIH0RkL/AzYLLb8VSRnwOfuh1EPdUK2FtsPYVaelGqq0SkHdAXWOZuJBUnIh4RWQ2kAgtU1ZFzqVfJQkQWisj6Uj5XAajqI6raGngbmOhutGdW3rn4yzwC5OM7nxorkHOppaSUbbX2jrWuEZEGwPvAPSVaFmoVVS1Q1T74WhAGiIgjTYT1aj4LVR0eYNF/A58AjzoYTqWUdy4icgtwBXCx1vCOqbP4e6ltUoDWxdbjgf0uxWKK8bfvvw+8raofuB1PVVDVTBH5EhgJVPlDCPXqzuJMRKRTsdXRwGa3YqksERkJPACMVtVst+Opx5KBTiKSICKhwDhgtssx1Xv+TuFXgE2q+me346kMEYktfNpRRCKA4Th07bKnofxE5H2gC74nb3YDd6jqPnejqhgR2QaEAen+TUtr8ZNdY4G/A7FAJrBaVUe4G1XgRGQU8BfAA7yqqn9wOaQKE5F3gJ/gG+H0EPCoqr7ialAVICIXAl8D6/D9fwd4WFXnuhdVxYhIL2A6vn9fQcBMVZ3iyHdZsjDGGFMea4YyxhhTLksWxhhjymXJwhhjTLksWRhjjCmXJQtjjDHlsmRhzFkQkazyS52x/iwRae9fbiAiL4rIdv+IoYtFJElEQv3L9eqlWVOzWbIwppqISHfAo6o7/Jum4RuYr5OqdgduBZr5Bx38HLjelUCNKYUlC2MqQHye9Y9htU5ErvdvDxKRf/rvFOaIyFwR+am/2s+Aj/3lOgBJwG9V1QvgH532E3/Zj/zljakR7DbXmIq5GugD9Mb3RnOyiCwGBgHtgJ5AHL7hr1/11xkEvONf7o7vbfSCMo6/HujvSOTGVIDdWRhTMRcC7/hH/DwEfIXv4n4h8J6qelX1IPBFsTotgLRADu5PIrn+yXmMcZ0lC2MqprThx8+0HeAkEO5f3gD0FpEz/R8MA3IqEJsxVc6ShTEVsxi43j/xTCwwBFiOb1rLa/x9F83xDbxXaBPQEUBVtwMrgMf9o6AiIp0K5/AQkRggTVXzquuEjDkTSxbGVMyHwFpgDbAIuN/f7PQ+vnks1uObN3wZcNRf5xNOTx6/BM4BtonIOuBlfpjvYihQ60ZBNXWXjTprTBUTkQaqmuW/O1gODFLVg/75Br7wr5fVsV14jA+Ah2rx/OOmjrGnoYypenP8E9KEAk/47zhQ1ZMi8ii+ebj3lFXZP1HSR5YoTE1idxbGGGPKZX0WxhhjymXJwhhjTLksWRhjjCmXJQtjjDHlsmRhjDGmXJYsjDHGlOv/AWAe3Bmq1Q33AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "test_means = grid.cv_results_[ 'mean_test_score' ]\n",
    "test_stds = grid.cv_results_[ 'std_test_score' ]\n",
    "train_means = grid.cv_results_[ 'mean_train_score' ]\n",
    "train_stds = grid.cv_results_[ 'std_train_score' ]\n",
    "\n",
    "# plot results\n",
    "n_Cs = len(Cs)\n",
    "number_penaltys = len(penaltys)\n",
    "test_scores =  np.array(test_means).reshape(n_Cs,number_penaltys)\n",
    "train_scores = np.array(train_means).reshape(n_Cs,number_penaltys)\n",
    "test_stds = np.array(test_stds).reshape(n_Cs,number_penaltys)\n",
    "train_stds = np.array(train_stds).reshape(n_Cs,number_penaltys)\n",
    "\n",
    "x_axis = np.log10(Cs)\n",
    "for i, value in enumerate(penaltys):\n",
    "    #pyplot.plot(log(Cs), test_scores[i], label= 'penalty:'   + str(value))\n",
    "    pyplot.errorbar(x_axis, test_scores[:,i], yerr=test_stds[:,i] ,label = penaltys[i] +' Test')\n",
    "    pyplot.errorbar(x_axis, train_scores[:,i], yerr=train_stds[:,i] ,label = penaltys[i] +' Train')\n",
    "    \n",
    "pyplot.legend()\n",
    "pyplot.xlabel( 'log(C)' )                                                                                                      \n",
    "pyplot.ylabel( 'accuary' )\n",
    "pyplot.savefig('LogisticGridSearchCV_C.png' )\n",
    "\n",
    "pyplot.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "best penalty is l2\n",
      "best C is 0.1\n"
     ]
    }
   ],
   "source": [
    "print(\"best penalty is\",grid.best_params_['penalty'])\n",
    "print(\"best C is\",grid.best_params_['C'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
